[==========] Running 56 tests from 4 test suites.
[----------] Global test environment set-up.
[----------] 30 tests from SecurityITest
[ RUN ] SecurityITest.TestAuthorizationOnListTablets
Loading random data
Initializing database '/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175691491353-13614-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 15:01:31 dist-test-slave-cnb0 krb5kdc[13621](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 15:01:31 dist-test-slave-cnb0 krb5kdc[13621](info): set up 2 sockets
Sep 23 15:01:31 dist-test-slave-cnb0 krb5kdc[13621](info): commencing operation
krb5kdc: starting...
WARNING: Logging before InitGoogleLogging() is written to STDERR
W20260923 15:01:32.544488 13614 mini_kdc.cc:121] Time spent starting KDC: real 1.030s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175691491353-13614-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175691491353-13614-0/krb5kdc/test-user.keytab.
Sep 23 15:01:32 dist-test-slave-cnb0 krb5kdc[13621](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175692, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T15:01:32Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T15:01:32Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.75.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.75.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.75.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.190.keytab.
Entry for principal kudu/127.13.75.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.190.keytab.
I20260923 15:01:32.679170 13614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKC4soy/build/release/bin/kudu
/tmp/dist-test-taskKC4soy/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175691491353-13614-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175691491353-13614-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175691491353-13614-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175691491353-13614-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.75.190:45865
--webserver_interface=127.13.75.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.190.keytab
--principal=kudu/127.13.75.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.75.148:39199
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.75.190:45865
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175691491353-13614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175691491353-13614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175691491353-13614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:01:32.765519 13634 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:01:32.765864 13634 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:01:32.765909 13634 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:01:32.768357 13634 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 15:01:32.768438 13634 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:01:32.768466 13634 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:01:32.768497 13634 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 15:01:32.768517 13634 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 15:01:32.771334 13634 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.75.148:39199
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175691491353-13614-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175691491353-13614-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.75.190:45865
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.190.keytab
--principal=kudu/127.13.75.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.75.190:45865
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175691491353-13614-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.75.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13634
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175691491353-13614-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:01:32.771658 13634 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:01:32.771947 13634 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:01:32.778056 13642 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:01:32.778020 13640 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:01:32.778035 13639 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:01:32.778191 13634 server_base.cc:1034] running on GCE node
I20260923 15:01:32.778676 13634 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:01:32.779004 13634 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:01:32.780156 13634 hybrid_clock.cc:648] HybridClock initialized: now 1790175692780141 us; error 30 us; skew 500 ppm
Sep 23 15:01:32 dist-test-slave-cnb0 krb5kdc[13621](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175692, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:01:32.782968 13634 init.cc:377] Logged in from keytab as kudu/127.13.75.190@KRBTEST.COM (short username kudu)
I20260923 15:01:32.783566 13634 webserver.cc:460] Webserver started at http://127.13.75.190:40325/ using document root <none> and password file <none>
I20260923 15:01:32.783798 13634 fs_manager.cc:359] Metadata directory not provided
I20260923 15:01:32.783844 13634 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:01:32.784011 13634 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:01:32.785220 13634 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175691491353-13614-0/minicluster-data/master-0/data/instance:
uuid: "0d15c771c1d14f6ab7ebdc1cc78361b2"
format_stamp: "Formatted at 2026-09-23 15:01:32 on dist-test-slave-cnb0"
server_key: "05dd2539fc21deb681d4f572d6bf9e79"
server_key_iv: "f3413262ff079f952766bceed757a6ec"
server_key_version: "encryptionkey@0"
I20260923 15:01:32.785701 13634 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175691491353-13614-0/minicluster-data/master-0/wal/instance:
uuid: "0d15c771c1d14f6ab7ebdc1cc78361b2"
format_stamp: "Formatted at 2026-09-23 15:01:32 on dist-test-slave-cnb0"
server_key: "05dd2539fc21deb681d4f572d6bf9e79"
server_key_iv: "f3413262ff079f952766bceed757a6ec"
server_key_version: "encryptionkey@0"
I20260923 15:01:32.787206 13634 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 15:01:32.788069 13649 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:01:32.788259 13634 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:01:32.788352 13634 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175691491353-13614-0/minicluster-data/master-0/data,/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175691491353-13614-0/minicluster-data/master-0/wal
uuid: "0d15c771c1d14f6ab7ebdc1cc78361b2"
format_stamp: "Formatted at 2026-09-23 15:01:32 on dist-test-slave-cnb0"
server_key: "05dd2539fc21deb681d4f572d6bf9e79"
server_key_iv: "f3413262ff079f952766bceed757a6ec"
server_key_version: "encryptionkey@0"
I20260923 15:01:32.788425 13634 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175691491353-13614-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175691491353-13614-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175691491353-13614-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:01:32.797019 13634 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:01:32.799535 13634 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:01:32.799716 13634 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:01:32.804631 13634 rpc_server.cc:307] RPC server started. Bound to: 127.13.75.190:45865
I20260923 15:01:32.804715 13701 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.75.190:45865 every 8 connection(s)
I20260923 15:01:32.805229 13634 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175691491353-13614-0/minicluster-data/master-0/data/info.pb
I20260923 15:01:32.806195 13702 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:01:32.808822 13702 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0d15c771c1d14f6ab7ebdc1cc78361b2: Bootstrap starting.
I20260923 15:01:32.809535 13702 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0d15c771c1d14f6ab7ebdc1cc78361b2: Neither blocks nor log segments found. Creating new log.
I20260923 15:01:32.809823 13702 log.cc:824] T 00000000000000000000000000000000 P 0d15c771c1d14f6ab7ebdc1cc78361b2: Log is configured to *not* fsync() on all Append() calls
I20260923 15:01:32.812935 13702 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0d15c771c1d14f6ab7ebdc1cc78361b2: No bootstrap required, opened a new log
I20260923 15:01:32.814817 13614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKC4soy/build/release/bin/kudu as pid 13634
I20260923 15:01:32.814970 13614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175691491353-13614-0/minicluster-data/master-0/wal/instance
I20260923 15:01:32.815049 13702 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0d15c771c1d14f6ab7ebdc1cc78361b2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d15c771c1d14f6ab7ebdc1cc78361b2" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 45865 } }
I20260923 15:01:32.815285 13702 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0d15c771c1d14f6ab7ebdc1cc78361b2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:01:32.815321 13702 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0d15c771c1d14f6ab7ebdc1cc78361b2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0d15c771c1d14f6ab7ebdc1cc78361b2, State: Initialized, Role: FOLLOWER
I20260923 15:01:32.815472 13702 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0d15c771c1d14f6ab7ebdc1cc78361b2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d15c771c1d14f6ab7ebdc1cc78361b2" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 45865 } }
I20260923 15:01:32.815552 13702 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0d15c771c1d14f6ab7ebdc1cc78361b2 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 15:01:32.815585 13702 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0d15c771c1d14f6ab7ebdc1cc78361b2 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 15:01:32.815637 13702 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0d15c771c1d14f6ab7ebdc1cc78361b2 [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:01:32.815814 13614 external_mini_cluster.cc:1428] Setting key 2ff70f13d60bf49cabfedf58fc95b453
I20260923 15:01:32.816392 13702 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0d15c771c1d14f6ab7ebdc1cc78361b2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d15c771c1d14f6ab7ebdc1cc78361b2" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 45865 } }
I20260923 15:01:32.816505 13702 leader_election.cc:302] T 00000000000000000000000000000000 P 0d15c771c1d14f6ab7ebdc1cc78361b2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0d15c771c1d14f6ab7ebdc1cc78361b2; no voters:
I20260923 15:01:32.816738 13702 leader_election.cc:288] T 00000000000000000000000000000000 P 0d15c771c1d14f6ab7ebdc1cc78361b2 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 15:01:32.816772 13705 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0d15c771c1d14f6ab7ebdc1cc78361b2 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 15:01:32.816956 13705 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0d15c771c1d14f6ab7ebdc1cc78361b2 [term 1 LEADER]: Becoming Leader. State: Replica: 0d15c771c1d14f6ab7ebdc1cc78361b2, State: Running, Role: LEADER
I20260923 15:01:32.817045 13702 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0d15c771c1d14f6ab7ebdc1cc78361b2 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 15:01:32.817143 13705 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0d15c771c1d14f6ab7ebdc1cc78361b2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d15c771c1d14f6ab7ebdc1cc78361b2" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 45865 } }
I20260923 15:01:32.817741 13709 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0d15c771c1d14f6ab7ebdc1cc78361b2 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0d15c771c1d14f6ab7ebdc1cc78361b2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d15c771c1d14f6ab7ebdc1cc78361b2" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 45865 } } }
I20260923 15:01:32.817829 13709 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0d15c771c1d14f6ab7ebdc1cc78361b2 [sys.catalog]: This master's current role is: LEADER
I20260923 15:01:32.818259 13716 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 15:01:32.818523 13707 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0d15c771c1d14f6ab7ebdc1cc78361b2 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0d15c771c1d14f6ab7ebdc1cc78361b2. Latest consensus state: current_term: 1 leader_uuid: "0d15c771c1d14f6ab7ebdc1cc78361b2" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0d15c771c1d14f6ab7ebdc1cc78361b2" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 45865 } } }
I20260923 15:01:32.818746 13707 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0d15c771c1d14f6ab7ebdc1cc78361b2 [sys.catalog]: This master's current role is: LEADER
I20260923 15:01:32.819100 13716 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 15:01:32.820941 13716 catalog_manager.cc:1347] Generated new cluster ID: cd13e61dab8d4513975b843d417bf2f9
I20260923 15:01:32.821002 13716 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 23 15:01:32 dist-test-slave-cnb0 krb5kdc[13621](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175692, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.75.190@KRBTEST.COM
I20260923 15:01:32.832165 13706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:32.816769 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51832 (local address 127.13.75.190:45865)
0923 15:01:32.817270 (+ 501us) server_negotiation.cc:206] Beginning negotiation
0923 15:01:32.817285 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0923 15:01:32.817731 (+ 446us) server_negotiation.cc:407] Connection header received
0923 15:01:32.820178 (+ 2447us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:01:32.820185 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:01:32.820242 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:01:32.820391 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:01:32.822072 (+ 1681us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:32.823126 (+ 1054us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:32.824218 (+ 1092us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:32.824638 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:32.824759 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:01:32.828232 (+ 3473us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:01:32.828236 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:01:32.828236 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:01:32.828247 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:01:32.830066 (+ 1819us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:01:32.830591 (+ 525us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:01:32.830595 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:01:32.830595 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:01:32.830697 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:01:32.831033 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:01:32.831034 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:01:32.831035 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:01:32.831303 (+ 268us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:01:32.831498 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0923 15:01:32.831759 (+ 261us) server_negotiation.cc:299] Negotiation successful
0923 15:01:32.831901 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":348,"thread_start_us":81,"threads_started":1}
I20260923 15:01:32.832856 13716 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 15:01:32.833539 13716 catalog_manager.cc:1504] Loading token signing keys...
I20260923 15:01:32.838055 13716 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0d15c771c1d14f6ab7ebdc1cc78361b2: Generated new TSK 0
I20260923 15:01:32.838236 13716 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.75.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.75.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.75.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.129.keytab.
Entry for principal kudu/127.13.75.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.129.keytab.
I20260923 15:01:32.869338 13614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKC4soy/build/release/bin/kudu
/tmp/dist-test-taskKC4soy/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175691491353-13614-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175691491353-13614-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175691491353-13614-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175691491353-13614-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.75.129:0
--local_ip_for_outbound_sockets=127.13.75.129
--webserver_interface=127.13.75.129
--webserver_port=0
--tserver_master_addrs=127.13.75.190:45865
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.129.keytab
--principal=kudu/127.13.75.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.75.148:39199
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175691491353-13614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175691491353-13614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175691491353-13614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:01:32.973308 13729 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:01:32.973536 13729 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:01:32.973563 13729 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:01:32.975024 13729 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:01:32.975069 13729 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:01:32.975099 13729 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.75.129
I20260923 15:01:32.976600 13729 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.75.148:39199
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175691491353-13614-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175691491353-13614-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.129.keytab
--principal=kudu/127.13.75.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.75.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175691491353-13614-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.75.129
--webserver_port=0
--tserver_master_addrs=127.13.75.190:45865
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13729
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.75.129
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175691491353-13614-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:01:32.976837 13729 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:01:32.977087 13729 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:01:32.982823 13737 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:01:32.982867 13735 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:01:32.983034 13734 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:01:32.983114 13729 server_base.cc:1034] running on GCE node
I20260923 15:01:32.983307 13729 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:01:32.983577 13729 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:01:32.984750 13729 hybrid_clock.cc:648] HybridClock initialized: now 1790175692984730 us; error 33 us; skew 500 ppm
Sep 23 15:01:32 dist-test-slave-cnb0 krb5kdc[13621](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175692, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:01:32.988047 13729 init.cc:377] Logged in from keytab as kudu/127.13.75.129@KRBTEST.COM (short username kudu)
I20260923 15:01:32.988492 13729 webserver.cc:460] Webserver started at http://127.13.75.129:44985/ using document root <none> and password file <none>
I20260923 15:01:32.988734 13729 fs_manager.cc:359] Metadata directory not provided
I20260923 15:01:32.988780 13729 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:01:32.988929 13729 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:01:32.989992 13729 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175691491353-13614-0/minicluster-data/ts-0/data/instance:
uuid: "79d00ea8e40343f8b6a8b040aa0382e0"
format_stamp: "Formatted at 2026-09-23 15:01:32 on dist-test-slave-cnb0"
server_key: "ab4e20b8f47bf45da7f339ab4d75cd77"
server_key_iv: "6557434918cb0613941ca0112b2d4dcf"
server_key_version: "encryptionkey@0"
I20260923 15:01:32.990347 13729 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175691491353-13614-0/minicluster-data/ts-0/wal/instance:
uuid: "79d00ea8e40343f8b6a8b040aa0382e0"
format_stamp: "Formatted at 2026-09-23 15:01:32 on dist-test-slave-cnb0"
server_key: "ab4e20b8f47bf45da7f339ab4d75cd77"
server_key_iv: "6557434918cb0613941ca0112b2d4dcf"
server_key_version: "encryptionkey@0"
I20260923 15:01:32.991853 13729 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260923 15:01:32.992646 13744 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:01:32.992813 13729 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260923 15:01:32.992900 13729 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175691491353-13614-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175691491353-13614-0/minicluster-data/ts-0/wal
uuid: "79d00ea8e40343f8b6a8b040aa0382e0"
format_stamp: "Formatted at 2026-09-23 15:01:32 on dist-test-slave-cnb0"
server_key: "ab4e20b8f47bf45da7f339ab4d75cd77"
server_key_iv: "6557434918cb0613941ca0112b2d4dcf"
server_key_version: "encryptionkey@0"
I20260923 15:01:32.992956 13729 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175691491353-13614-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175691491353-13614-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175691491353-13614-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:01:33.005486 13729 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:01:33.008083 13729 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:01:33.008206 13729 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:01:33.008422 13729 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 15:01:33.008756 13729 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 15:01:33.008792 13729 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 15:01:33.008834 13729 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 15:01:33.008860 13729 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 15:01:33.014990 13729 rpc_server.cc:307] RPC server started. Bound to: 127.13.75.129:37691
I20260923 15:01:33.015066 13857 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.75.129:37691 every 8 connection(s)
I20260923 15:01:33.015422 13729 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175691491353-13614-0/minicluster-data/ts-0/data/info.pb
Sep 23 15:01:33 dist-test-slave-cnb0 krb5kdc[13621](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175692, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.129@KRBTEST.COM for kudu/127.13.75.190@KRBTEST.COM
I20260923 15:01:33.025257 13614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKC4soy/build/release/bin/kudu as pid 13729
I20260923 15:01:33.025334 13614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175691491353-13614-0/minicluster-data/ts-0/wal/instance
I20260923 15:01:33.025496 13614 external_mini_cluster.cc:1428] Setting key 81640a92de51de778dd91381675fe75d
I20260923 15:01:33.025714 13706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:33.016241 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.129:53277 (local address 127.13.75.190:45865)
0923 15:01:33.016319 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0923 15:01:33.016321 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:01:33.016564 (+ 243us) server_negotiation.cc:407] Connection header received
0923 15:01:33.017089 (+ 525us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:01:33.017090 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:01:33.017104 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:01:33.017174 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:01:33.018518 (+ 1344us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:33.019087 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:33.020207 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:33.020413 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:33.020484 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:01:33.022611 (+ 2127us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:01:33.022612 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:01:33.022613 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:01:33.022620 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:01:33.024142 (+ 1522us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:01:33.024578 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:01:33.024579 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:01:33.024579 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:01:33.024633 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:01:33.024920 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:01:33.024921 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:01:33.024921 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:01:33.025026 (+ 105us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:01:33.025100 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0923 15:01:33.025505 (+ 405us) server_negotiation.cc:299] Negotiation successful
0923 15:01:33.025618 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260923 15:01:33.025964 13860 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:33.016215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:45865 (local address 127.13.75.129:53277)
0923 15:01:33.016411 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0923 15:01:33.016459 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0923 15:01:33.016995 (+ 536us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:01:33.017324 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:01:33.017328 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:01:33.018093 (+ 765us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:01:33.018414 (+ 321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:01:33.018420 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:01:33.019187 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:01:33.019188 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:01:33.020100 (+ 912us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:01:33.020102 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:01:33.020515 (+ 413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:01:33.020517 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:01:33.020627 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:01:33.021117 (+ 490us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:01:33.021125 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:01:33.022493 (+ 1368us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:01:33.024227 (+ 1734us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:01:33.024234 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:01:33.024236 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:01:33.024506 (+ 270us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:01:33.024714 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:01:33.024715 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:01:33.024715 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:01:33.024825 (+ 110us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:01:33.025138 (+ 313us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:01:33.025144 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:01:33.025275 (+ 131us) client_negotiation.cc:769] Sending connection context
0923 15:01:33.025474 (+ 199us) client_negotiation.cc:240] Negotiation successful
0923 15:01:33.025650 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":102,"threads_started":1}
I20260923 15:01:33.026463 13858 heartbeater.cc:347] Connected to a master server at 127.13.75.190:45865
I20260923 15:01:33.026618 13858 heartbeater.cc:464] Registering TS with master...
I20260923 15:01:33.026820 13858 heartbeater.cc:511] Master 127.13.75.190:45865 requested a full tablet report, sending...
I20260923 15:01:33.027379 13666 ts_manager.cc:194] Registered new tserver with Master: 79d00ea8e40343f8b6a8b040aa0382e0 (127.13.75.129:37691)
I20260923 15:01:33.028306 13666 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.75.129@KRBTEST.COM'} at 127.13.75.129:53277
WARNING: no policy specified for kudu/127.13.75.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.75.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.75.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.130.keytab.
Entry for principal kudu/127.13.75.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.130.keytab.
I20260923 15:01:33.051918 13614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKC4soy/build/release/bin/kudu
/tmp/dist-test-taskKC4soy/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175691491353-13614-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175691491353-13614-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175691491353-13614-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175691491353-13614-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.75.130:0
--local_ip_for_outbound_sockets=127.13.75.130
--webserver_interface=127.13.75.130
--webserver_port=0
--tserver_master_addrs=127.13.75.190:45865
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.130.keytab
--principal=kudu/127.13.75.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.75.148:39199
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175691491353-13614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175691491353-13614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175691491353-13614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:01:33.133456 13863 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:01:33.133667 13863 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:01:33.133693 13863 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:01:33.134790 13863 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:01:33.134824 13863 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:01:33.134850 13863 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.75.130
I20260923 15:01:33.136252 13863 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.75.148:39199
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175691491353-13614-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175691491353-13614-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.130.keytab
--principal=kudu/127.13.75.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.75.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175691491353-13614-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.75.130
--webserver_port=0
--tserver_master_addrs=127.13.75.190:45865
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13863
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.75.130
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175691491353-13614-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:01:33.136442 13863 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:01:33.136617 13863 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:01:33.141515 13869 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:01:33.141638 13871 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:01:33.141515 13868 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:01:33.141781 13863 server_base.cc:1034] running on GCE node
I20260923 15:01:33.142004 13863 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:01:33.142251 13863 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:01:33.143452 13863 hybrid_clock.cc:648] HybridClock initialized: now 1790175693143443 us; error 35 us; skew 500 ppm
Sep 23 15:01:33 dist-test-slave-cnb0 krb5kdc[13621](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175693, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:01:33.146000 13863 init.cc:377] Logged in from keytab as kudu/127.13.75.130@KRBTEST.COM (short username kudu)
I20260923 15:01:33.146536 13863 webserver.cc:460] Webserver started at http://127.13.75.130:36677/ using document root <none> and password file <none>
I20260923 15:01:33.146757 13863 fs_manager.cc:359] Metadata directory not provided
I20260923 15:01:33.146819 13863 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:01:33.146991 13863 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:01:33.148048 13863 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175691491353-13614-0/minicluster-data/ts-1/data/instance:
uuid: "9e82fad0b5c647bea155f6d80439621d"
format_stamp: "Formatted at 2026-09-23 15:01:33 on dist-test-slave-cnb0"
server_key: "84a00c692c86ffd6a3665195a8c14420"
server_key_iv: "d1d6cb1899a0006948f993cd5bfc6cdc"
server_key_version: "encryptionkey@0"
I20260923 15:01:33.148417 13863 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175691491353-13614-0/minicluster-data/ts-1/wal/instance:
uuid: "9e82fad0b5c647bea155f6d80439621d"
format_stamp: "Formatted at 2026-09-23 15:01:33 on dist-test-slave-cnb0"
server_key: "84a00c692c86ffd6a3665195a8c14420"
server_key_iv: "d1d6cb1899a0006948f993cd5bfc6cdc"
server_key_version: "encryptionkey@0"
I20260923 15:01:33.149791 13863 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 15:01:33.150736 13878 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:01:33.150981 13863 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:01:33.151096 13863 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175691491353-13614-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175691491353-13614-0/minicluster-data/ts-1/wal
uuid: "9e82fad0b5c647bea155f6d80439621d"
format_stamp: "Formatted at 2026-09-23 15:01:33 on dist-test-slave-cnb0"
server_key: "84a00c692c86ffd6a3665195a8c14420"
server_key_iv: "d1d6cb1899a0006948f993cd5bfc6cdc"
server_key_version: "encryptionkey@0"
I20260923 15:01:33.151180 13863 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175691491353-13614-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175691491353-13614-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175691491353-13614-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:01:33.163908 13863 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:01:33.165949 13863 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:01:33.166111 13863 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:01:33.166342 13863 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 15:01:33.166705 13863 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 15:01:33.166756 13863 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 15:01:33.166798 13863 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 15:01:33.166826 13863 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 15:01:33.172668 13863 rpc_server.cc:307] RPC server started. Bound to: 127.13.75.130:44129
I20260923 15:01:33.172719 13991 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.75.130:44129 every 8 connection(s)
I20260923 15:01:33.173079 13863 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175691491353-13614-0/minicluster-data/ts-1/data/info.pb
I20260923 15:01:33.176872 13614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKC4soy/build/release/bin/kudu as pid 13863
I20260923 15:01:33.176988 13614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175691491353-13614-0/minicluster-data/ts-1/wal/instance
I20260923 15:01:33.177155 13614 external_mini_cluster.cc:1428] Setting key ae8a264306acd5fc894c7bbf82eb6e0a
Sep 23 15:01:33 dist-test-slave-cnb0 krb5kdc[13621](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175693, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.130@KRBTEST.COM for kudu/127.13.75.190@KRBTEST.COM
I20260923 15:01:33.184055 13706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:33.173827 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.130:55015 (local address 127.13.75.190:45865)
0923 15:01:33.173907 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0923 15:01:33.173909 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:01:33.174116 (+ 207us) server_negotiation.cc:407] Connection header received
0923 15:01:33.174621 (+ 505us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:01:33.174622 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:01:33.174632 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:01:33.174691 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:01:33.175737 (+ 1046us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:33.176269 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:33.177409 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:33.177701 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:33.177774 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:01:33.180452 (+ 2678us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:01:33.180454 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:01:33.180454 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:01:33.180463 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:01:33.182271 (+ 1808us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:01:33.182742 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:01:33.182745 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:01:33.182745 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:01:33.182820 (+ 75us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:01:33.183186 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:01:33.183187 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:01:33.183187 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:01:33.183377 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:01:33.183463 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0923 15:01:33.183838 (+ 375us) server_negotiation.cc:299] Negotiation successful
0923 15:01:33.183936 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260923 15:01:33.184306 13994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:33.173786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:45865 (local address 127.13.75.130:55015)
0923 15:01:33.173966 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0923 15:01:33.174024 (+ 58us) client_negotiation.cc:174] Beginning negotiation
0923 15:01:33.174505 (+ 481us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:01:33.174748 (+ 243us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:01:33.174752 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:01:33.175400 (+ 648us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:01:33.175609 (+ 209us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:01:33.175614 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:01:33.176362 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:01:33.176363 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:01:33.177318 (+ 955us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:01:33.177321 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:01:33.177811 (+ 490us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:01:33.177813 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:01:33.177994 (+ 181us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:01:33.178708 (+ 714us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:01:33.178717 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:01:33.180347 (+ 1630us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:01:33.182381 (+ 2034us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:01:33.182389 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:01:33.182391 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:01:33.182653 (+ 262us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:01:33.182976 (+ 323us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:01:33.182977 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:01:33.182978 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:01:33.183110 (+ 132us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:01:33.183499 (+ 389us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:01:33.183510 (+ 11us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:01:33.183668 (+ 158us) client_negotiation.cc:769] Sending connection context
0923 15:01:33.183772 (+ 104us) client_negotiation.cc:240] Negotiation successful
0923 15:01:33.183926 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":101,"threads_started":1}
I20260923 15:01:33.184903 13992 heartbeater.cc:347] Connected to a master server at 127.13.75.190:45865
I20260923 15:01:33.185026 13992 heartbeater.cc:464] Registering TS with master...
I20260923 15:01:33.185245 13992 heartbeater.cc:511] Master 127.13.75.190:45865 requested a full tablet report, sending...
I20260923 15:01:33.185725 13666 ts_manager.cc:194] Registered new tserver with Master: 9e82fad0b5c647bea155f6d80439621d (127.13.75.130:44129)
I20260923 15:01:33.186156 13666 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.75.130@KRBTEST.COM'} at 127.13.75.130:55015
WARNING: no policy specified for kudu/127.13.75.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.75.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.75.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.131.keytab.
Entry for principal kudu/127.13.75.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.131.keytab.
I20260923 15:01:33.206689 13614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKC4soy/build/release/bin/kudu
/tmp/dist-test-taskKC4soy/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175691491353-13614-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175691491353-13614-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175691491353-13614-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175691491353-13614-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.75.131:0
--local_ip_for_outbound_sockets=127.13.75.131
--webserver_interface=127.13.75.131
--webserver_port=0
--tserver_master_addrs=127.13.75.190:45865
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.131.keytab
--principal=kudu/127.13.75.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.75.148:39199
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175691491353-13614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175691491353-13614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175691491353-13614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:01:33.290189 13997 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:01:33.290418 13997 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:01:33.290444 13997 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:01:33.291764 13997 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:01:33.291811 13997 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:01:33.291841 13997 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.75.131
I20260923 15:01:33.293342 13997 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.75.148:39199
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175691491353-13614-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175691491353-13614-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.131.keytab
--principal=kudu/127.13.75.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.75.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175691491353-13614-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.75.131
--webserver_port=0
--tserver_master_addrs=127.13.75.190:45865
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.13997
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.75.131
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175691491353-13614-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:01:33.293596 13997 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:01:33.293828 13997 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:01:33.299014 14005 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:01:33.299024 14002 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:01:33.299069 13997 server_base.cc:1034] running on GCE node
W20260923 15:01:33.299032 14003 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:01:33.299418 13997 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:01:33.299660 13997 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:01:33.300843 13997 hybrid_clock.cc:648] HybridClock initialized: now 1790175693300836 us; error 47 us; skew 500 ppm
Sep 23 15:01:33 dist-test-slave-cnb0 krb5kdc[13621](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175693, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:01:33.303372 13997 init.cc:377] Logged in from keytab as kudu/127.13.75.131@KRBTEST.COM (short username kudu)
I20260923 15:01:33.303882 13997 webserver.cc:460] Webserver started at http://127.13.75.131:40651/ using document root <none> and password file <none>
I20260923 15:01:33.304112 13997 fs_manager.cc:359] Metadata directory not provided
I20260923 15:01:33.304180 13997 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:01:33.304327 13997 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:01:33.305368 13997 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175691491353-13614-0/minicluster-data/ts-2/data/instance:
uuid: "a363343748384eceaaf2f6a3b86eea17"
format_stamp: "Formatted at 2026-09-23 15:01:33 on dist-test-slave-cnb0"
server_key: "bb905a29b6fab83e34137b17236303f7"
server_key_iv: "e9a3bb8859552e5a8ca5f2ecc151b0ca"
server_key_version: "encryptionkey@0"
I20260923 15:01:33.305735 13997 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175691491353-13614-0/minicluster-data/ts-2/wal/instance:
uuid: "a363343748384eceaaf2f6a3b86eea17"
format_stamp: "Formatted at 2026-09-23 15:01:33 on dist-test-slave-cnb0"
server_key: "bb905a29b6fab83e34137b17236303f7"
server_key_iv: "e9a3bb8859552e5a8ca5f2ecc151b0ca"
server_key_version: "encryptionkey@0"
I20260923 15:01:33.307305 13997 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260923 15:01:33.308230 14012 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:01:33.308471 13997 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:01:33.308578 13997 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175691491353-13614-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175691491353-13614-0/minicluster-data/ts-2/wal
uuid: "a363343748384eceaaf2f6a3b86eea17"
format_stamp: "Formatted at 2026-09-23 15:01:33 on dist-test-slave-cnb0"
server_key: "bb905a29b6fab83e34137b17236303f7"
server_key_iv: "e9a3bb8859552e5a8ca5f2ecc151b0ca"
server_key_version: "encryptionkey@0"
I20260923 15:01:33.308660 13997 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175691491353-13614-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175691491353-13614-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175691491353-13614-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:01:33.324820 13997 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:01:33.326704 13997 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:01:33.326862 13997 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:01:33.327148 13997 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 15:01:33.327474 13997 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 15:01:33.327526 13997 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 15:01:33.327579 13997 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 15:01:33.327606 13997 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 15:01:33.333859 13997 rpc_server.cc:307] RPC server started. Bound to: 127.13.75.131:44355
I20260923 15:01:33.333901 14125 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.75.131:44355 every 8 connection(s)
I20260923 15:01:33.334227 13997 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175691491353-13614-0/minicluster-data/ts-2/data/info.pb
Sep 23 15:01:33 dist-test-slave-cnb0 krb5kdc[13621](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175693, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.131@KRBTEST.COM for kudu/127.13.75.190@KRBTEST.COM
I20260923 15:01:33.342334 13614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKC4soy/build/release/bin/kudu as pid 13997
I20260923 15:01:33.342427 13614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnListTablets.1790175691491353-13614-0/minicluster-data/ts-2/wal/instance
I20260923 15:01:33.342604 13614 external_mini_cluster.cc:1428] Setting key 91ba70039cd092141e39513d094929dd
I20260923 15:01:33.343941 14128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:33.334965 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:45865 (local address 127.13.75.131:53353)
0923 15:01:33.335121 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0923 15:01:33.335157 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0923 15:01:33.335619 (+ 462us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:01:33.335812 (+ 193us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:01:33.335816 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:01:33.336379 (+ 563us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:01:33.336582 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:01:33.336586 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:01:33.337254 (+ 668us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:01:33.337255 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:01:33.338158 (+ 903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:01:33.338161 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:01:33.338613 (+ 452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:01:33.338614 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:01:33.338751 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:01:33.339263 (+ 512us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:01:33.339271 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:01:33.340671 (+ 1400us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:01:33.342232 (+ 1561us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:01:33.342239 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:01:33.342241 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:01:33.342626 (+ 385us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:01:33.342858 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:01:33.342859 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:01:33.342859 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:01:33.343015 (+ 156us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:01:33.343324 (+ 309us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:01:33.343331 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:01:33.343505 (+ 174us) client_negotiation.cc:769] Sending connection context
0923 15:01:33.343623 (+ 118us) client_negotiation.cc:240] Negotiation successful
0923 15:01:33.343771 (+ 148us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":93,"threads_started":1}
I20260923 15:01:33.343964 13706 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:33.334985 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.131:53353 (local address 127.13.75.190:45865)
0923 15:01:33.335040 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0923 15:01:33.335042 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:01:33.335239 (+ 197us) server_negotiation.cc:407] Connection header received
0923 15:01:33.335698 (+ 459us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:01:33.335699 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:01:33.335708 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:01:33.335760 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:01:33.336668 (+ 908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:33.337178 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:33.338254 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:33.338527 (+ 273us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:33.338604 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:01:33.340769 (+ 2165us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:01:33.340770 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:01:33.340771 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:01:33.340777 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:01:33.342147 (+ 1370us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:01:33.342723 (+ 576us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:01:33.342724 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:01:33.342724 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:01:33.342788 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:01:33.343100 (+ 312us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:01:33.343101 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:01:33.343102 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:01:33.343250 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:01:33.343322 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0923 15:01:33.343797 (+ 475us) server_negotiation.cc:299] Negotiation successful
0923 15:01:33.343902 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260923 15:01:33.344406 14126 heartbeater.cc:347] Connected to a master server at 127.13.75.190:45865
I20260923 15:01:33.344503 14126 heartbeater.cc:464] Registering TS with master...
I20260923 15:01:33.344678 14126 heartbeater.cc:511] Master 127.13.75.190:45865 requested a full tablet report, sending...
I20260923 15:01:33.345062 13666 ts_manager.cc:194] Registered new tserver with Master: a363343748384eceaaf2f6a3b86eea17 (127.13.75.131:44355)
I20260923 15:01:33.345431 13666 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.75.131@KRBTEST.COM'} at 127.13.75.131:53353
I20260923 15:01:33.354647 13614 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 23 15:01:33 dist-test-slave-cnb0 krb5kdc[13621](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175693, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 23 15:01:33 dist-test-slave-cnb0 krb5kdc[13621](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175693, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.13.75.129@KRBTEST.COM
I20260923 15:01:33.378396 14133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:33.370147 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50042 (local address 127.13.75.129:37691)
0923 15:01:33.370398 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0923 15:01:33.370403 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 15:01:33.370414 (+ 11us) server_negotiation.cc:407] Connection header received
0923 15:01:33.370441 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:01:33.370442 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:01:33.370457 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:01:33.370536 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:01:33.371272 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:33.372049 (+ 777us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:33.372805 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:33.373150 (+ 345us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:33.373217 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:01:33.375754 (+ 2537us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:01:33.375763 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:01:33.375763 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:01:33.375770 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:01:33.377203 (+ 1433us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:01:33.377540 (+ 337us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:01:33.377543 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:01:33.377543 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:01:33.377597 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:01:33.377761 (+ 164us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:01:33.377761 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:01:33.377761 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:01:33.377892 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:01:33.378013 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0923 15:01:33.378076 (+ 63us) server_negotiation.cc:299] Negotiation successful
0923 15:01:33.378135 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":202,"thread_start_us":99,"threads_started":1}
W20260923 15:01:33.378777 13772 server_base.cc:1116] Unauthorized access attempt to method kudu.tserver.TabletServerService.ListTablets from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:50042
Sep 23 15:01:33 dist-test-slave-cnb0 krb5kdc[13621](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175693, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 23 15:01:33 dist-test-slave-cnb0 krb5kdc[13621](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175693, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.75.129@KRBTEST.COM
I20260923 15:01:33.402267 14133 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:33.394671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50058 (local address 127.13.75.129:37691)
0923 15:01:33.394775 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0923 15:01:33.394778 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:01:33.394855 (+ 77us) server_negotiation.cc:407] Connection header received
0923 15:01:33.395020 (+ 165us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:01:33.395021 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:01:33.395034 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:01:33.395097 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:01:33.395826 (+ 729us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:33.396354 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:33.397175 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:33.397392 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:33.397454 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:01:33.399747 (+ 2293us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:01:33.399748 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:01:33.399749 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:01:33.399755 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:01:33.401175 (+ 1420us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:01:33.401490 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:01:33.401491 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:01:33.401491 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:01:33.401551 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:01:33.401782 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:01:33.401782 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:01:33.401783 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:01:33.401869 (+ 86us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:01:33.401962 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0923 15:01:33.402105 (+ 143us) server_negotiation.cc:299] Negotiation successful
0923 15:01:33.402171 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":72}
I20260923 15:01:33.403018 13614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKC4soy/build/release/bin/kudu with pid 13729
I20260923 15:01:33.408545 13614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKC4soy/build/release/bin/kudu with pid 13863
I20260923 15:01:33.413792 13614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKC4soy/build/release/bin/kudu with pid 13997
I20260923 15:01:33.418264 13614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKC4soy/build/release/bin/kudu with pid 13634
2026-09-23T15:01:33Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnListTablets (1933 ms)
[ RUN ] SecurityITest.TestAuthorizationOnChecksum
Loading random data
Initializing database '/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175691491353-13614-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 15:01:33 dist-test-slave-cnb0 krb5kdc[14140](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 15:01:33 dist-test-slave-cnb0 krb5kdc[14140](info): set up 2 sockets
Sep 23 15:01:33 dist-test-slave-cnb0 krb5kdc[14140](info): commencing operation
krb5kdc: starting...
W20260923 15:01:34.463837 13614 mini_kdc.cc:121] Time spent starting KDC: real 1.024s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175691491353-13614-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175691491353-13614-0/krb5kdc/test-user.keytab.
Sep 23 15:01:34 dist-test-slave-cnb0 krb5kdc[14140](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175694, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T15:01:34Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T15:01:34Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.75.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.75.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.75.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.190.keytab.
Entry for principal kudu/127.13.75.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.190.keytab.
I20260923 15:01:34.587227 13614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKC4soy/build/release/bin/kudu
/tmp/dist-test-taskKC4soy/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175691491353-13614-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175691491353-13614-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175691491353-13614-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175691491353-13614-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.75.190:44465
--webserver_interface=127.13.75.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.190.keytab
--principal=kudu/127.13.75.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.75.148:45135
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.75.190:44465
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175691491353-13614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175691491353-13614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175691491353-13614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:01:34.679539 14153 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:01:34.679770 14153 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:01:34.679811 14153 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:01:34.681036 14153 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 15:01:34.681097 14153 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:01:34.681123 14153 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:01:34.681155 14153 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 15:01:34.681183 14153 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 15:01:34.683696 14153 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.75.148:45135
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175691491353-13614-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175691491353-13614-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.75.190:44465
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.190.keytab
--principal=kudu/127.13.75.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.75.190:44465
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175691491353-13614-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.75.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14153
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175691491353-13614-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:01:34.683996 14153 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:01:34.684297 14153 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:01:34.689549 14161 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:01:34.689636 14158 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:01:34.689610 14159 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:01:34.689828 14153 server_base.cc:1034] running on GCE node
I20260923 15:01:34.690096 14153 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:01:34.690402 14153 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:01:34.691570 14153 hybrid_clock.cc:648] HybridClock initialized: now 1790175694691558 us; error 33 us; skew 500 ppm
Sep 23 15:01:34 dist-test-slave-cnb0 krb5kdc[14140](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175694, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:01:34.694000 14153 init.cc:377] Logged in from keytab as kudu/127.13.75.190@KRBTEST.COM (short username kudu)
I20260923 15:01:34.694453 14153 webserver.cc:460] Webserver started at http://127.13.75.190:36335/ using document root <none> and password file <none>
I20260923 15:01:34.694658 14153 fs_manager.cc:359] Metadata directory not provided
I20260923 15:01:34.694700 14153 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:01:34.694787 14153 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:01:34.695871 14153 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175691491353-13614-0/minicluster-data/master-0/data/instance:
uuid: "8641003eb86e4fa89c8e3d72854ab0dd"
format_stamp: "Formatted at 2026-09-23 15:01:34 on dist-test-slave-cnb0"
server_key: "2aea7a4a464197e17bc8791b8164aaa3"
server_key_iv: "cbe54e7739477121dd13a0a68e485422"
server_key_version: "encryptionkey@0"
I20260923 15:01:34.696167 14153 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175691491353-13614-0/minicluster-data/master-0/wal/instance:
uuid: "8641003eb86e4fa89c8e3d72854ab0dd"
format_stamp: "Formatted at 2026-09-23 15:01:34 on dist-test-slave-cnb0"
server_key: "2aea7a4a464197e17bc8791b8164aaa3"
server_key_iv: "cbe54e7739477121dd13a0a68e485422"
server_key_version: "encryptionkey@0"
I20260923 15:01:34.697468 14153 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 15:01:34.698149 14168 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:01:34.698318 14153 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 15:01:34.698403 14153 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175691491353-13614-0/minicluster-data/master-0/data,/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175691491353-13614-0/minicluster-data/master-0/wal
uuid: "8641003eb86e4fa89c8e3d72854ab0dd"
format_stamp: "Formatted at 2026-09-23 15:01:34 on dist-test-slave-cnb0"
server_key: "2aea7a4a464197e17bc8791b8164aaa3"
server_key_iv: "cbe54e7739477121dd13a0a68e485422"
server_key_version: "encryptionkey@0"
I20260923 15:01:34.698462 14153 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175691491353-13614-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175691491353-13614-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175691491353-13614-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:01:34.709362 14153 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:01:34.711431 14153 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:01:34.711544 14153 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:01:34.715380 14153 rpc_server.cc:307] RPC server started. Bound to: 127.13.75.190:44465
I20260923 15:01:34.715444 14220 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.75.190:44465 every 8 connection(s)
I20260923 15:01:34.715776 14153 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175691491353-13614-0/minicluster-data/master-0/data/info.pb
I20260923 15:01:34.716665 14221 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:01:34.718935 14221 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8641003eb86e4fa89c8e3d72854ab0dd: Bootstrap starting.
I20260923 15:01:34.719524 14221 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 8641003eb86e4fa89c8e3d72854ab0dd: Neither blocks nor log segments found. Creating new log.
I20260923 15:01:34.719823 14221 log.cc:824] T 00000000000000000000000000000000 P 8641003eb86e4fa89c8e3d72854ab0dd: Log is configured to *not* fsync() on all Append() calls
I20260923 15:01:34.720568 14221 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 8641003eb86e4fa89c8e3d72854ab0dd: No bootstrap required, opened a new log
I20260923 15:01:34.722131 14221 raft_consensus.cc:348] T 00000000000000000000000000000000 P 8641003eb86e4fa89c8e3d72854ab0dd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8641003eb86e4fa89c8e3d72854ab0dd" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 44465 } }
I20260923 15:01:34.722293 14221 raft_consensus.cc:374] T 00000000000000000000000000000000 P 8641003eb86e4fa89c8e3d72854ab0dd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:01:34.722340 14221 raft_consensus.cc:729] T 00000000000000000000000000000000 P 8641003eb86e4fa89c8e3d72854ab0dd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8641003eb86e4fa89c8e3d72854ab0dd, State: Initialized, Role: FOLLOWER
I20260923 15:01:34.722520 14221 consensus_queue.cc:260] T 00000000000000000000000000000000 P 8641003eb86e4fa89c8e3d72854ab0dd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8641003eb86e4fa89c8e3d72854ab0dd" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 44465 } }
I20260923 15:01:34.722599 13614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKC4soy/build/release/bin/kudu as pid 14153
I20260923 15:01:34.722626 14221 raft_consensus.cc:388] T 00000000000000000000000000000000 P 8641003eb86e4fa89c8e3d72854ab0dd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 15:01:34.722669 14221 raft_consensus.cc:482] T 00000000000000000000000000000000 P 8641003eb86e4fa89c8e3d72854ab0dd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 15:01:34.722681 13614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175691491353-13614-0/minicluster-data/master-0/wal/instance
I20260923 15:01:34.722723 14221 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 8641003eb86e4fa89c8e3d72854ab0dd [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:01:34.722831 13614 external_mini_cluster.cc:1428] Setting key 00c050606c6bbdcb51e25331ab4e8089
I20260923 15:01:34.723405 14221 raft_consensus.cc:504] T 00000000000000000000000000000000 P 8641003eb86e4fa89c8e3d72854ab0dd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8641003eb86e4fa89c8e3d72854ab0dd" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 44465 } }
I20260923 15:01:34.723528 14221 leader_election.cc:302] T 00000000000000000000000000000000 P 8641003eb86e4fa89c8e3d72854ab0dd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 8641003eb86e4fa89c8e3d72854ab0dd; no voters:
I20260923 15:01:34.723721 14221 leader_election.cc:288] T 00000000000000000000000000000000 P 8641003eb86e4fa89c8e3d72854ab0dd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 15:01:34.723745 14226 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 8641003eb86e4fa89c8e3d72854ab0dd [term 1 FOLLOWER]: Leader election won for term 1
I20260923 15:01:34.723886 14226 raft_consensus.cc:686] T 00000000000000000000000000000000 P 8641003eb86e4fa89c8e3d72854ab0dd [term 1 LEADER]: Becoming Leader. State: Replica: 8641003eb86e4fa89c8e3d72854ab0dd, State: Running, Role: LEADER
I20260923 15:01:34.723996 14221 sys_catalog.cc:564] T 00000000000000000000000000000000 P 8641003eb86e4fa89c8e3d72854ab0dd [sys.catalog]: configured and running, proceeding with master startup.
I20260923 15:01:34.723977 14226 consensus_queue.cc:237] T 00000000000000000000000000000000 P 8641003eb86e4fa89c8e3d72854ab0dd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8641003eb86e4fa89c8e3d72854ab0dd" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 44465 } }
I20260923 15:01:34.724360 14226 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8641003eb86e4fa89c8e3d72854ab0dd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 8641003eb86e4fa89c8e3d72854ab0dd. Latest consensus state: current_term: 1 leader_uuid: "8641003eb86e4fa89c8e3d72854ab0dd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8641003eb86e4fa89c8e3d72854ab0dd" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 44465 } } }
I20260923 15:01:34.724436 14226 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8641003eb86e4fa89c8e3d72854ab0dd [sys.catalog]: This master's current role is: LEADER
I20260923 15:01:34.724530 14228 sys_catalog.cc:455] T 00000000000000000000000000000000 P 8641003eb86e4fa89c8e3d72854ab0dd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "8641003eb86e4fa89c8e3d72854ab0dd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "8641003eb86e4fa89c8e3d72854ab0dd" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 44465 } } }
I20260923 15:01:34.724606 14228 sys_catalog.cc:458] T 00000000000000000000000000000000 P 8641003eb86e4fa89c8e3d72854ab0dd [sys.catalog]: This master's current role is: LEADER
W20260923 15:01:34.726316 14242 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 8641003eb86e4fa89c8e3d72854ab0dd: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260923 15:01:34.726402 14242 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260923 15:01:34.726488 14230 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 15:01:34.726642 14230 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 15:01:34.728538 14230 catalog_manager.cc:1347] Generated new cluster ID: 638d7e9a7fd04edfb5c0e3d92c751065
I20260923 15:01:34.728596 14230 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 23 15:01:34 dist-test-slave-cnb0 krb5kdc[14140](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175694, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.75.190@KRBTEST.COM
I20260923 15:01:34.734737 14225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:34.723378 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44066 (local address 127.13.75.190:44465)
0923 15:01:34.723629 (+ 251us) server_negotiation.cc:206] Beginning negotiation
0923 15:01:34.723640 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0923 15:01:34.723674 (+ 34us) server_negotiation.cc:407] Connection header received
0923 15:01:34.723812 (+ 138us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:01:34.723820 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:01:34.723887 (+ 67us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:01:34.724092 (+ 205us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:01:34.724920 (+ 828us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:34.726266 (+ 1346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:34.727393 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:34.727904 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:34.728045 (+ 141us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:01:34.730845 (+ 2800us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:01:34.730850 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:01:34.730850 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:01:34.730864 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:01:34.732981 (+ 2117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:01:34.733387 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:01:34.733391 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:01:34.733392 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:01:34.733531 (+ 139us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:01:34.733729 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:01:34.733730 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:01:34.733730 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:01:34.734074 (+ 344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:01:34.734263 (+ 189us) server_negotiation.cc:1035] Waiting for connection context
0923 15:01:34.734342 (+ 79us) server_negotiation.cc:299] Negotiation successful
0923 15:01:34.734508 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":197,"thread_start_us":95,"threads_started":1}
I20260923 15:01:34.744347 14230 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 15:01:34.744899 14230 catalog_manager.cc:1504] Loading token signing keys...
I20260923 15:01:34.750630 14230 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 8641003eb86e4fa89c8e3d72854ab0dd: Generated new TSK 0
I20260923 15:01:34.750831 14230 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.75.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.75.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.75.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.129.keytab.
Entry for principal kudu/127.13.75.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.129.keytab.
I20260923 15:01:34.781066 13614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKC4soy/build/release/bin/kudu
/tmp/dist-test-taskKC4soy/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175691491353-13614-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175691491353-13614-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175691491353-13614-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175691491353-13614-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.75.129:0
--local_ip_for_outbound_sockets=127.13.75.129
--webserver_interface=127.13.75.129
--webserver_port=0
--tserver_master_addrs=127.13.75.190:44465
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.129.keytab
--principal=kudu/127.13.75.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.75.148:45135
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175691491353-13614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175691491353-13614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175691491353-13614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:01:34.876912 14247 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:01:34.877125 14247 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:01:34.877151 14247 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:01:34.878352 14247 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:01:34.878394 14247 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:01:34.878429 14247 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.75.129
I20260923 15:01:34.879980 14247 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.75.148:45135
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175691491353-13614-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175691491353-13614-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.129.keytab
--principal=kudu/127.13.75.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.75.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175691491353-13614-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.75.129
--webserver_port=0
--tserver_master_addrs=127.13.75.190:44465
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14247
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.75.129
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175691491353-13614-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:01:34.880234 14247 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:01:34.880486 14247 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:01:34.885582 14255 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:01:34.885604 14247 server_base.cc:1034] running on GCE node
W20260923 15:01:34.885615 14253 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:01:34.885584 14252 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:01:34.886067 14247 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:01:34.886341 14247 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:01:34.887506 14247 hybrid_clock.cc:648] HybridClock initialized: now 1790175694887495 us; error 37 us; skew 500 ppm
Sep 23 15:01:34 dist-test-slave-cnb0 krb5kdc[14140](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175694, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:01:34.890359 14247 init.cc:377] Logged in from keytab as kudu/127.13.75.129@KRBTEST.COM (short username kudu)
I20260923 15:01:34.890841 14247 webserver.cc:460] Webserver started at http://127.13.75.129:46243/ using document root <none> and password file <none>
I20260923 15:01:34.891112 14247 fs_manager.cc:359] Metadata directory not provided
I20260923 15:01:34.891181 14247 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:01:34.891340 14247 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:01:34.892408 14247 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175691491353-13614-0/minicluster-data/ts-0/data/instance:
uuid: "6c9724ac708540a0924d6b26b3dba6c6"
format_stamp: "Formatted at 2026-09-23 15:01:34 on dist-test-slave-cnb0"
server_key: "fbbc71b78c5715e0e34298f2ed2238b9"
server_key_iv: "ea9566778ca28831926fd973b9b49e5e"
server_key_version: "encryptionkey@0"
I20260923 15:01:34.892772 14247 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175691491353-13614-0/minicluster-data/ts-0/wal/instance:
uuid: "6c9724ac708540a0924d6b26b3dba6c6"
format_stamp: "Formatted at 2026-09-23 15:01:34 on dist-test-slave-cnb0"
server_key: "fbbc71b78c5715e0e34298f2ed2238b9"
server_key_iv: "ea9566778ca28831926fd973b9b49e5e"
server_key_version: "encryptionkey@0"
I20260923 15:01:34.894176 14247 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 15:01:34.895074 14262 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:01:34.895251 14247 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:01:34.895359 14247 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175691491353-13614-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175691491353-13614-0/minicluster-data/ts-0/wal
uuid: "6c9724ac708540a0924d6b26b3dba6c6"
format_stamp: "Formatted at 2026-09-23 15:01:34 on dist-test-slave-cnb0"
server_key: "fbbc71b78c5715e0e34298f2ed2238b9"
server_key_iv: "ea9566778ca28831926fd973b9b49e5e"
server_key_version: "encryptionkey@0"
I20260923 15:01:34.895432 14247 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175691491353-13614-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175691491353-13614-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175691491353-13614-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:01:34.905435 14247 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:01:34.907706 14247 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:01:34.907862 14247 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:01:34.908092 14247 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 15:01:34.908396 14247 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 15:01:34.908447 14247 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 15:01:34.908488 14247 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 15:01:34.908514 14247 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 15:01:34.914793 14247 rpc_server.cc:307] RPC server started. Bound to: 127.13.75.129:33671
I20260923 15:01:34.914835 14375 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.75.129:33671 every 8 connection(s)
I20260923 15:01:34.915210 14247 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175691491353-13614-0/minicluster-data/ts-0/data/info.pb
I20260923 15:01:34.916687 13614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKC4soy/build/release/bin/kudu as pid 14247
I20260923 15:01:34.916783 13614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175691491353-13614-0/minicluster-data/ts-0/wal/instance
I20260923 15:01:34.916949 13614 external_mini_cluster.cc:1428] Setting key d1965b9da67d3fcac968b2d8c7081293
Sep 23 15:01:34 dist-test-slave-cnb0 krb5kdc[14140](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175694, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.129@KRBTEST.COM for kudu/127.13.75.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.13.75.130@KRBTEST.COM; defaulting to no policy
I20260923 15:01:34.925859 14225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:34.916000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.129:56701 (local address 127.13.75.190:44465)
0923 15:01:34.916092 (+ 92us) server_negotiation.cc:206] Beginning negotiation
0923 15:01:34.916095 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:01:34.916273 (+ 178us) server_negotiation.cc:407] Connection header received
0923 15:01:34.917445 (+ 1172us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:01:34.917446 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:01:34.917461 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:01:34.917519 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:01:34.918159 (+ 640us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:34.918717 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:34.919937 (+ 1220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:34.920299 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:34.920371 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:01:34.922696 (+ 2325us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:01:34.922698 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:01:34.922699 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:01:34.922711 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:01:34.924409 (+ 1698us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:01:34.924823 (+ 414us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:01:34.924824 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:01:34.924825 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:01:34.924891 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:01:34.925188 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:01:34.925190 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:01:34.925190 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:01:34.925334 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:01:34.925391 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0923 15:01:34.925722 (+ 331us) server_negotiation.cc:299] Negotiation successful
0923 15:01:34.925792 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260923 15:01:34.926076 14378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:34.915952 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:44465 (local address 127.13.75.129:56701)
0923 15:01:34.916143 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0923 15:01:34.916192 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0923 15:01:34.917348 (+ 1156us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:01:34.917597 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:01:34.917601 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:01:34.917867 (+ 266us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:01:34.918060 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:01:34.918063 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:01:34.918814 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:01:34.918815 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:01:34.919851 (+ 1036us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:01:34.919854 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:01:34.920389 (+ 535us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:01:34.920390 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:01:34.920531 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:01:34.921179 (+ 648us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:01:34.921192 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:01:34.922599 (+ 1407us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:01:34.924523 (+ 1924us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:01:34.924530 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:01:34.924531 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:01:34.924748 (+ 217us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:01:34.924976 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:01:34.924977 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:01:34.924978 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:01:34.925124 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:01:34.925403 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:01:34.925410 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:01:34.925582 (+ 172us) client_negotiation.cc:769] Sending connection context
0923 15:01:34.925694 (+ 112us) client_negotiation.cc:240] Negotiation successful
0923 15:01:34.925819 (+ 125us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":98,"threads_started":1}
I20260923 15:01:34.926568 14376 heartbeater.cc:347] Connected to a master server at 127.13.75.190:44465
I20260923 15:01:34.926678 14376 heartbeater.cc:464] Registering TS with master...
I20260923 15:01:34.926857 14376 heartbeater.cc:511] Master 127.13.75.190:44465 requested a full tablet report, sending...
I20260923 15:01:34.927399 14185 ts_manager.cc:194] Registered new tserver with Master: 6c9724ac708540a0924d6b26b3dba6c6 (127.13.75.129:33671)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.75.130@KRBTEST.COM" created.
I20260923 15:01:34.928082 14185 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.75.129@KRBTEST.COM'} at 127.13.75.129:56701
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.75.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.130.keytab.
Entry for principal kudu/127.13.75.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.130.keytab.
I20260923 15:01:34.942585 13614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKC4soy/build/release/bin/kudu
/tmp/dist-test-taskKC4soy/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175691491353-13614-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175691491353-13614-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175691491353-13614-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175691491353-13614-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.75.130:0
--local_ip_for_outbound_sockets=127.13.75.130
--webserver_interface=127.13.75.130
--webserver_port=0
--tserver_master_addrs=127.13.75.190:44465
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.130.keytab
--principal=kudu/127.13.75.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.75.148:45135
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175691491353-13614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175691491353-13614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175691491353-13614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:01:35.026728 14381 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:01:35.027014 14381 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:01:35.027064 14381 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:01:35.028416 14381 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:01:35.028509 14381 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:01:35.028560 14381 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.75.130
I20260923 15:01:35.030236 14381 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.75.148:45135
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175691491353-13614-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175691491353-13614-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.130.keytab
--principal=kudu/127.13.75.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.75.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175691491353-13614-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.75.130
--webserver_port=0
--tserver_master_addrs=127.13.75.190:44465
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14381
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.75.130
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175691491353-13614-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:01:35.030545 14381 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:01:35.030825 14381 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:01:35.036633 14389 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:01:35.036623 14387 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:01:35.036619 14386 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:01:35.036741 14381 server_base.cc:1034] running on GCE node
I20260923 15:01:35.037078 14381 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:01:35.037377 14381 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:01:35.038556 14381 hybrid_clock.cc:648] HybridClock initialized: now 1790175695038540 us; error 36 us; skew 500 ppm
Sep 23 15:01:35 dist-test-slave-cnb0 krb5kdc[14140](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175695, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:01:35.041473 14381 init.cc:377] Logged in from keytab as kudu/127.13.75.130@KRBTEST.COM (short username kudu)
I20260923 15:01:35.042024 14381 webserver.cc:460] Webserver started at http://127.13.75.130:38711/ using document root <none> and password file <none>
I20260923 15:01:35.042289 14381 fs_manager.cc:359] Metadata directory not provided
I20260923 15:01:35.042338 14381 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:01:35.042513 14381 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:01:35.043720 14381 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175691491353-13614-0/minicluster-data/ts-1/data/instance:
uuid: "5ee313ff56e741b8a374762dbbbba570"
format_stamp: "Formatted at 2026-09-23 15:01:35 on dist-test-slave-cnb0"
server_key: "ddfd588d70d527f8670f6bc0d5a4c0ca"
server_key_iv: "82e6164aec3e77a47b8fa92f7df9ecae"
server_key_version: "encryptionkey@0"
I20260923 15:01:35.044107 14381 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175691491353-13614-0/minicluster-data/ts-1/wal/instance:
uuid: "5ee313ff56e741b8a374762dbbbba570"
format_stamp: "Formatted at 2026-09-23 15:01:35 on dist-test-slave-cnb0"
server_key: "ddfd588d70d527f8670f6bc0d5a4c0ca"
server_key_iv: "82e6164aec3e77a47b8fa92f7df9ecae"
server_key_version: "encryptionkey@0"
I20260923 15:01:35.045584 14381 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260923 15:01:35.046466 14396 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:01:35.046692 14381 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260923 15:01:35.046768 14381 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175691491353-13614-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175691491353-13614-0/minicluster-data/ts-1/wal
uuid: "5ee313ff56e741b8a374762dbbbba570"
format_stamp: "Formatted at 2026-09-23 15:01:35 on dist-test-slave-cnb0"
server_key: "ddfd588d70d527f8670f6bc0d5a4c0ca"
server_key_iv: "82e6164aec3e77a47b8fa92f7df9ecae"
server_key_version: "encryptionkey@0"
I20260923 15:01:35.046860 14381 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175691491353-13614-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175691491353-13614-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175691491353-13614-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:01:35.067508 14381 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:01:35.069587 14381 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:01:35.069741 14381 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:01:35.070026 14381 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 15:01:35.070374 14381 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 15:01:35.070429 14381 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 15:01:35.070472 14381 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 15:01:35.070500 14381 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 15:01:35.076411 14381 rpc_server.cc:307] RPC server started. Bound to: 127.13.75.130:45777
I20260923 15:01:35.076460 14509 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.75.130:45777 every 8 connection(s)
I20260923 15:01:35.076812 14381 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175691491353-13614-0/minicluster-data/ts-1/data/info.pb
I20260923 15:01:35.078018 13614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKC4soy/build/release/bin/kudu as pid 14381
I20260923 15:01:35.078090 13614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175691491353-13614-0/minicluster-data/ts-1/wal/instance
I20260923 15:01:35.078265 13614 external_mini_cluster.cc:1428] Setting key f7d772a75aff0dd24d2541eaff8eeae0
Sep 23 15:01:35 dist-test-slave-cnb0 krb5kdc[14140](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175695, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.130@KRBTEST.COM for kudu/127.13.75.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.13.75.131@KRBTEST.COM; defaulting to no policy
I20260923 15:01:35.086630 14225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:35.077577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.130:46725 (local address 127.13.75.190:44465)
0923 15:01:35.077660 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0923 15:01:35.077663 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:01:35.077874 (+ 211us) server_negotiation.cc:407] Connection header received
0923 15:01:35.078908 (+ 1034us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:01:35.078910 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:01:35.078923 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:01:35.078970 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:01:35.079631 (+ 661us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:35.080123 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:35.081143 (+ 1020us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:35.081348 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:35.081420 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:01:35.083693 (+ 2273us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:01:35.083694 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:01:35.083694 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:01:35.083703 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:01:35.085110 (+ 1407us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:01:35.085545 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:01:35.085546 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:01:35.085546 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:01:35.085607 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:01:35.085862 (+ 255us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:01:35.085863 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:01:35.085864 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:01:35.086022 (+ 158us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:01:35.086098 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0923 15:01:35.086442 (+ 344us) server_negotiation.cc:299] Negotiation successful
0923 15:01:35.086522 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260923 15:01:35.086771 14512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:35.077545 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:44465 (local address 127.13.75.130:46725)
0923 15:01:35.077711 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0923 15:01:35.077753 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0923 15:01:35.078768 (+ 1015us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:01:35.079066 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:01:35.079070 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:01:35.079335 (+ 265us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:01:35.079535 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:01:35.079540 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:01:35.080218 (+ 678us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:01:35.080219 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:01:35.081051 (+ 832us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:01:35.081053 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:01:35.081440 (+ 387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:01:35.081441 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:01:35.081572 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:01:35.082175 (+ 603us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:01:35.082184 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:01:35.083592 (+ 1408us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:01:35.085204 (+ 1612us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:01:35.085211 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:01:35.085214 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:01:35.085450 (+ 236us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:01:35.085672 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:01:35.085673 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:01:35.085674 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:01:35.085790 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:01:35.086101 (+ 311us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:01:35.086108 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:01:35.086300 (+ 192us) client_negotiation.cc:769] Sending connection context
0923 15:01:35.086411 (+ 111us) client_negotiation.cc:240] Negotiation successful
0923 15:01:35.086552 (+ 141us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":89,"threads_started":1}
I20260923 15:01:35.087498 14510 heartbeater.cc:347] Connected to a master server at 127.13.75.190:44465
I20260923 15:01:35.087607 14510 heartbeater.cc:464] Registering TS with master...
I20260923 15:01:35.087811 14510 heartbeater.cc:511] Master 127.13.75.190:44465 requested a full tablet report, sending...
I20260923 15:01:35.088300 14185 ts_manager.cc:194] Registered new tserver with Master: 5ee313ff56e741b8a374762dbbbba570 (127.13.75.130:45777)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.75.131@KRBTEST.COM" created.
I20260923 15:01:35.088811 14185 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.75.130@KRBTEST.COM'} at 127.13.75.130:46725
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.75.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.131.keytab.
Entry for principal kudu/127.13.75.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.131.keytab.
I20260923 15:01:35.105250 13614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKC4soy/build/release/bin/kudu
/tmp/dist-test-taskKC4soy/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175691491353-13614-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175691491353-13614-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175691491353-13614-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175691491353-13614-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.75.131:0
--local_ip_for_outbound_sockets=127.13.75.131
--webserver_interface=127.13.75.131
--webserver_port=0
--tserver_master_addrs=127.13.75.190:44465
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.131.keytab
--principal=kudu/127.13.75.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.75.148:45135
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_enforce_access_control with env {KRB5CCNAME=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175691491353-13614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175691491353-13614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175691491353-13614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:01:35.188912 14515 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:01:35.189179 14515 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:01:35.189224 14515 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:01:35.190485 14515 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:01:35.190559 14515 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:01:35.190609 14515 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.75.131
I20260923 15:01:35.192241 14515 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.75.148:45135
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175691491353-13614-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175691491353-13614-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.131.keytab
--principal=kudu/127.13.75.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.75.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175691491353-13614-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.75.131
--webserver_port=0
--tserver_master_addrs=127.13.75.190:44465
--tserver_enforce_access_control=true
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14515
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.75.131
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175691491353-13614-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:01:35.192543 14515 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:01:35.192785 14515 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:01:35.198712 14520 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:01:35.198699 14521 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:01:35.198925 14515 server_base.cc:1034] running on GCE node
W20260923 15:01:35.199103 14523 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:01:35.199582 14515 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:01:35.199919 14515 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:01:35.201071 14515 hybrid_clock.cc:648] HybridClock initialized: now 1790175695201047 us; error 45 us; skew 500 ppm
Sep 23 15:01:35 dist-test-slave-cnb0 krb5kdc[14140](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175695, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:01:35.205173 14515 init.cc:377] Logged in from keytab as kudu/127.13.75.131@KRBTEST.COM (short username kudu)
I20260923 15:01:35.205776 14515 webserver.cc:460] Webserver started at http://127.13.75.131:33703/ using document root <none> and password file <none>
I20260923 15:01:35.206261 14515 fs_manager.cc:359] Metadata directory not provided
I20260923 15:01:35.206329 14515 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:01:35.206475 14515 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:01:35.208124 14515 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175691491353-13614-0/minicluster-data/ts-2/data/instance:
uuid: "40feb304df974a3c989b1a15725c3503"
format_stamp: "Formatted at 2026-09-23 15:01:35 on dist-test-slave-cnb0"
server_key: "ae0c31d669581e5a7a8d65d9b23a9134"
server_key_iv: "dc79e34fd2d08dbb90475c17305d41d8"
server_key_version: "encryptionkey@0"
I20260923 15:01:35.208555 14515 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175691491353-13614-0/minicluster-data/ts-2/wal/instance:
uuid: "40feb304df974a3c989b1a15725c3503"
format_stamp: "Formatted at 2026-09-23 15:01:35 on dist-test-slave-cnb0"
server_key: "ae0c31d669581e5a7a8d65d9b23a9134"
server_key_iv: "dc79e34fd2d08dbb90475c17305d41d8"
server_key_version: "encryptionkey@0"
I20260923 15:01:35.211022 14515 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20260923 15:01:35.212241 14530 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:01:35.212647 14515 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:01:35.212751 14515 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175691491353-13614-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175691491353-13614-0/minicluster-data/ts-2/wal
uuid: "40feb304df974a3c989b1a15725c3503"
format_stamp: "Formatted at 2026-09-23 15:01:35 on dist-test-slave-cnb0"
server_key: "ae0c31d669581e5a7a8d65d9b23a9134"
server_key_iv: "dc79e34fd2d08dbb90475c17305d41d8"
server_key_version: "encryptionkey@0"
I20260923 15:01:35.212847 14515 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175691491353-13614-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175691491353-13614-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175691491353-13614-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:01:35.262567 14515 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:01:35.264991 14515 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:01:35.265168 14515 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:01:35.265424 14515 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 15:01:35.265795 14515 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 15:01:35.265857 14515 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 15:01:35.265900 14515 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 15:01:35.265926 14515 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 15:01:35.272030 14515 rpc_server.cc:307] RPC server started. Bound to: 127.13.75.131:43969
I20260923 15:01:35.272068 14643 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.75.131:43969 every 8 connection(s)
I20260923 15:01:35.272411 14515 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175691491353-13614-0/minicluster-data/ts-2/data/info.pb
Sep 23 15:01:35 dist-test-slave-cnb0 krb5kdc[14140](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175695, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.131@KRBTEST.COM for kudu/127.13.75.190@KRBTEST.COM
I20260923 15:01:35.281095 13614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKC4soy/build/release/bin/kudu as pid 14515
I20260923 15:01:35.281214 13614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizationOnChecksum.1790175691491353-13614-0/minicluster-data/ts-2/wal/instance
I20260923 15:01:35.281401 13614 external_mini_cluster.cc:1428] Setting key 84261bfc4372347050a74ff39810bb1e
I20260923 15:01:35.282961 14225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:35.273146 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.131:40277 (local address 127.13.75.190:44465)
0923 15:01:35.273235 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0923 15:01:35.273238 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:01:35.273396 (+ 158us) server_negotiation.cc:407] Connection header received
0923 15:01:35.274358 (+ 962us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:01:35.274359 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:01:35.274371 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:01:35.274434 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:01:35.275167 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:35.275713 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:35.276858 (+ 1145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:35.277051 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:35.277112 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:01:35.279701 (+ 2589us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:01:35.279703 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:01:35.279703 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:01:35.279712 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:01:35.281196 (+ 1484us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:01:35.281735 (+ 539us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:01:35.281737 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:01:35.281738 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:01:35.281808 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:01:35.282092 (+ 284us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:01:35.282093 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:01:35.282093 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:01:35.282273 (+ 180us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:01:35.282341 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0923 15:01:35.282745 (+ 404us) server_negotiation.cc:299] Negotiation successful
0923 15:01:35.282827 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260923 15:01:35.283004 14646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:35.273106 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:44465 (local address 127.13.75.131:40277)
0923 15:01:35.273268 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0923 15:01:35.273306 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0923 15:01:35.274216 (+ 910us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:01:35.274497 (+ 281us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:01:35.274501 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:01:35.274765 (+ 264us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:01:35.275048 (+ 283us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:01:35.275053 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:01:35.275820 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:01:35.275821 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:01:35.276761 (+ 940us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:01:35.276763 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:01:35.277155 (+ 392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:01:35.277156 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:01:35.277281 (+ 125us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:01:35.277924 (+ 643us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:01:35.277933 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:01:35.279585 (+ 1652us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:01:35.281310 (+ 1725us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:01:35.281319 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:01:35.281322 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:01:35.281651 (+ 329us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:01:35.281875 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:01:35.281876 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:01:35.281877 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:01:35.282006 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:01:35.282342 (+ 336us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:01:35.282348 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:01:35.282511 (+ 163us) client_negotiation.cc:769] Sending connection context
0923 15:01:35.282619 (+ 108us) client_negotiation.cc:240] Negotiation successful
0923 15:01:35.282765 (+ 146us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":97,"threads_started":1}
I20260923 15:01:35.283555 14644 heartbeater.cc:347] Connected to a master server at 127.13.75.190:44465
I20260923 15:01:35.283663 14644 heartbeater.cc:464] Registering TS with master...
I20260923 15:01:35.283855 14644 heartbeater.cc:511] Master 127.13.75.190:44465 requested a full tablet report, sending...
I20260923 15:01:35.284241 14185 ts_manager.cc:194] Registered new tserver with Master: 40feb304df974a3c989b1a15725c3503 (127.13.75.131:43969)
I20260923 15:01:35.284632 14185 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.75.131@KRBTEST.COM'} at 127.13.75.131:40277
I20260923 15:01:35.293474 13614 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260923 15:01:35.301846 14225 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:35.295216 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44080 (local address 127.13.75.190:44465)
0923 15:01:35.295291 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0923 15:01:35.295294 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:01:35.295383 (+ 89us) server_negotiation.cc:407] Connection header received
0923 15:01:35.295477 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:01:35.295478 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:01:35.295496 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:01:35.295552 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:01:35.296205 (+ 653us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:35.296762 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:35.297612 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:35.297811 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:35.297887 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:01:35.299284 (+ 1397us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:01:35.299285 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:01:35.299286 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:01:35.299294 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:01:35.300535 (+ 1241us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:01:35.300935 (+ 400us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:01:35.300937 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:01:35.300938 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:01:35.301004 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:01:35.301206 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:01:35.301207 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:01:35.301207 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:01:35.301368 (+ 161us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:01:35.301484 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0923 15:01:35.301641 (+ 157us) server_negotiation.cc:299] Negotiation successful
0923 15:01:35.301732 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260923 15:01:35.304217 14185 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44080:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 15:01:35.305908 14185 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 15:01:35.313371 14655 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:35.308494 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.129:33671 (local address 127.0.0.1:41140)
0923 15:01:35.308822 (+ 328us) negotiation.cc:107] Waiting for socket to connect
0923 15:01:35.308867 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0923 15:01:35.308940 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:01:35.309289 (+ 349us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:01:35.309290 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:01:35.309292 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:01:35.309371 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:01:35.309373 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:01:35.310691 (+ 1318us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:01:35.310692 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:01:35.312157 (+ 1465us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:01:35.312159 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:01:35.313056 (+ 897us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:01:35.313057 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:01:35.313139 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:01:35.313154 (+ 15us) client_negotiation.cc:769] Sending connection context
0923 15:01:35.313233 (+ 79us) client_negotiation.cc:240] Negotiation successful
0923 15:01:35.313288 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":300,"mutex_wait_us":38,"thread_start_us":72,"threads_started":1}
I20260923 15:01:35.313687 14658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:35.308649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41140 (local address 127.13.75.129:33671)
0923 15:01:35.308824 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0923 15:01:35.308829 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 15:01:35.309025 (+ 196us) server_negotiation.cc:407] Connection header received
0923 15:01:35.309104 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:01:35.309107 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:01:35.309123 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:01:35.309197 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:01:35.309453 (+ 256us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:35.310576 (+ 1123us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:35.312258 (+ 1682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:35.312972 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:35.313027 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:01:35.313077 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0923 15:01:35.313261 (+ 184us) server_negotiation.cc:299] Negotiation successful
0923 15:01:35.313303 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":82,"threads_started":1}
I20260923 15:01:35.314265 14656 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:35.308536 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.131:43969 (local address 127.0.0.1:44524)
0923 15:01:35.308871 (+ 335us) negotiation.cc:107] Waiting for socket to connect
0923 15:01:35.308880 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 15:01:35.308940 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:01:35.309286 (+ 346us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:01:35.309289 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:01:35.309291 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:01:35.309367 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:01:35.309369 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:01:35.311775 (+ 2406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:01:35.311776 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:01:35.312730 (+ 954us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:01:35.312733 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:01:35.313981 (+ 1248us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:01:35.313982 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:01:35.314044 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:01:35.314048 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 15:01:35.314120 (+ 72us) client_negotiation.cc:240] Negotiation successful
0923 15:01:35.314164 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":318,"thread_start_us":72,"threads_started":1}
I20260923 15:01:35.314356 14660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:35.308902 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53990 (local address 127.13.75.130:45777)
0923 15:01:35.309304 (+ 402us) server_negotiation.cc:206] Beginning negotiation
0923 15:01:35.309308 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 15:01:35.309317 (+ 9us) server_negotiation.cc:407] Connection header received
0923 15:01:35.309355 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:01:35.309357 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:01:35.309373 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:01:35.309451 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:01:35.309653 (+ 202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:35.310683 (+ 1030us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:35.313110 (+ 2427us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:35.314018 (+ 908us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:35.314105 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:01:35.314158 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0923 15:01:35.314241 (+ 83us) server_negotiation.cc:299] Negotiation successful
0923 15:01:35.314280 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":363,"thread_start_us":112,"threads_started":1}
I20260923 15:01:35.314381 14310 tablet_service.cc:1467] Processing CreateTablet for tablet fe1f1f411a084ed9bb043b0f8c446cba (DEFAULT_TABLE table=test-table [id=8b7f789c63874045ab5768db3ac421a0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 15:01:35.314601 14657 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:35.308602 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44524 (local address 127.13.75.131:43969)
0923 15:01:35.308902 (+ 300us) server_negotiation.cc:206] Beginning negotiation
0923 15:01:35.308908 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 15:01:35.308922 (+ 14us) server_negotiation.cc:407] Connection header received
0923 15:01:35.309057 (+ 135us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:01:35.309060 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:01:35.309080 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:01:35.309172 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:01:35.310709 (+ 1537us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:35.311673 (+ 964us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:35.312819 (+ 1146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:35.313774 (+ 955us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:35.313875 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:01:35.313939 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0923 15:01:35.314472 (+ 533us) server_negotiation.cc:299] Negotiation successful
0923 15:01:35.314521 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":255,"thread_start_us":108,"threads_started":1}
I20260923 15:01:35.314678 14659 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:35.308747 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.130:45777 (local address 127.0.0.1:53990)
0923 15:01:35.309115 (+ 368us) negotiation.cc:107] Waiting for socket to connect
0923 15:01:35.309126 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 15:01:35.309172 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:01:35.309539 (+ 367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:01:35.309540 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:01:35.309542 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:01:35.309577 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:01:35.309578 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:01:35.310784 (+ 1206us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:01:35.310785 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:01:35.312155 (+ 1370us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:01:35.312159 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:01:35.314113 (+ 1954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:01:35.314114 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:01:35.314169 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:01:35.314173 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 15:01:35.314230 (+ 57us) client_negotiation.cc:240] Negotiation successful
0923 15:01:35.314270 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":344,"mutex_wait_us":12,"thread_start_us":74,"threads_started":1}
I20260923 15:01:35.314805 14310 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fe1f1f411a084ed9bb043b0f8c446cba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:01:35.315346 14578 tablet_service.cc:1467] Processing CreateTablet for tablet fe1f1f411a084ed9bb043b0f8c446cba (DEFAULT_TABLE table=test-table [id=8b7f789c63874045ab5768db3ac421a0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 15:01:35.315466 14444 tablet_service.cc:1467] Processing CreateTablet for tablet fe1f1f411a084ed9bb043b0f8c446cba (DEFAULT_TABLE table=test-table [id=8b7f789c63874045ab5768db3ac421a0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 15:01:35.315721 14578 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fe1f1f411a084ed9bb043b0f8c446cba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:01:35.315779 14444 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fe1f1f411a084ed9bb043b0f8c446cba. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:01:35.317735 14661 tablet_bootstrap.cc:492] T fe1f1f411a084ed9bb043b0f8c446cba P 6c9724ac708540a0924d6b26b3dba6c6: Bootstrap starting.
I20260923 15:01:35.318533 14661 tablet_bootstrap.cc:654] T fe1f1f411a084ed9bb043b0f8c446cba P 6c9724ac708540a0924d6b26b3dba6c6: Neither blocks nor log segments found. Creating new log.
I20260923 15:01:35.319094 14661 log.cc:824] T fe1f1f411a084ed9bb043b0f8c446cba P 6c9724ac708540a0924d6b26b3dba6c6: Log is configured to *not* fsync() on all Append() calls
I20260923 15:01:35.319365 14663 tablet_bootstrap.cc:492] T fe1f1f411a084ed9bb043b0f8c446cba P 5ee313ff56e741b8a374762dbbbba570: Bootstrap starting.
I20260923 15:01:35.319576 14662 tablet_bootstrap.cc:492] T fe1f1f411a084ed9bb043b0f8c446cba P 40feb304df974a3c989b1a15725c3503: Bootstrap starting.
I20260923 15:01:35.320235 14661 tablet_bootstrap.cc:492] T fe1f1f411a084ed9bb043b0f8c446cba P 6c9724ac708540a0924d6b26b3dba6c6: No bootstrap required, opened a new log
I20260923 15:01:35.320252 14662 tablet_bootstrap.cc:654] T fe1f1f411a084ed9bb043b0f8c446cba P 40feb304df974a3c989b1a15725c3503: Neither blocks nor log segments found. Creating new log.
I20260923 15:01:35.320315 14661 ts_tablet_manager.cc:1397] T fe1f1f411a084ed9bb043b0f8c446cba P 6c9724ac708540a0924d6b26b3dba6c6: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260923 15:01:35.320814 14662 log.cc:824] T fe1f1f411a084ed9bb043b0f8c446cba P 40feb304df974a3c989b1a15725c3503: Log is configured to *not* fsync() on all Append() calls
I20260923 15:01:35.321750 14663 tablet_bootstrap.cc:654] T fe1f1f411a084ed9bb043b0f8c446cba P 5ee313ff56e741b8a374762dbbbba570: Neither blocks nor log segments found. Creating new log.
I20260923 15:01:35.322281 14663 log.cc:824] T fe1f1f411a084ed9bb043b0f8c446cba P 5ee313ff56e741b8a374762dbbbba570: Log is configured to *not* fsync() on all Append() calls
I20260923 15:01:35.322608 14661 raft_consensus.cc:348] T fe1f1f411a084ed9bb043b0f8c446cba P 6c9724ac708540a0924d6b26b3dba6c6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c9724ac708540a0924d6b26b3dba6c6" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 33671 } } peers { permanent_uuid: "5ee313ff56e741b8a374762dbbbba570" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 45777 } } peers { permanent_uuid: "40feb304df974a3c989b1a15725c3503" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 43969 } }
I20260923 15:01:35.322796 14661 raft_consensus.cc:374] T fe1f1f411a084ed9bb043b0f8c446cba P 6c9724ac708540a0924d6b26b3dba6c6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:01:35.322844 14661 raft_consensus.cc:729] T fe1f1f411a084ed9bb043b0f8c446cba P 6c9724ac708540a0924d6b26b3dba6c6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6c9724ac708540a0924d6b26b3dba6c6, State: Initialized, Role: FOLLOWER
I20260923 15:01:35.322857 14662 tablet_bootstrap.cc:492] T fe1f1f411a084ed9bb043b0f8c446cba P 40feb304df974a3c989b1a15725c3503: No bootstrap required, opened a new log
I20260923 15:01:35.322942 14662 ts_tablet_manager.cc:1397] T fe1f1f411a084ed9bb043b0f8c446cba P 40feb304df974a3c989b1a15725c3503: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260923 15:01:35.323014 14661 consensus_queue.cc:260] T fe1f1f411a084ed9bb043b0f8c446cba P 6c9724ac708540a0924d6b26b3dba6c6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c9724ac708540a0924d6b26b3dba6c6" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 33671 } } peers { permanent_uuid: "5ee313ff56e741b8a374762dbbbba570" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 45777 } } peers { permanent_uuid: "40feb304df974a3c989b1a15725c3503" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 43969 } }
I20260923 15:01:35.323084 14663 tablet_bootstrap.cc:492] T fe1f1f411a084ed9bb043b0f8c446cba P 5ee313ff56e741b8a374762dbbbba570: No bootstrap required, opened a new log
I20260923 15:01:35.323161 14663 ts_tablet_manager.cc:1397] T fe1f1f411a084ed9bb043b0f8c446cba P 5ee313ff56e741b8a374762dbbbba570: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260923 15:01:35.323311 14661 ts_tablet_manager.cc:1428] T fe1f1f411a084ed9bb043b0f8c446cba P 6c9724ac708540a0924d6b26b3dba6c6: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260923 15:01:35.323438 14376 heartbeater.cc:503] Master 127.13.75.190:44465 was elected leader, sending a full tablet report...
I20260923 15:01:35.325166 14662 raft_consensus.cc:348] T fe1f1f411a084ed9bb043b0f8c446cba P 40feb304df974a3c989b1a15725c3503 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c9724ac708540a0924d6b26b3dba6c6" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 33671 } } peers { permanent_uuid: "5ee313ff56e741b8a374762dbbbba570" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 45777 } } peers { permanent_uuid: "40feb304df974a3c989b1a15725c3503" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 43969 } }
I20260923 15:01:35.325181 14663 raft_consensus.cc:348] T fe1f1f411a084ed9bb043b0f8c446cba P 5ee313ff56e741b8a374762dbbbba570 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c9724ac708540a0924d6b26b3dba6c6" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 33671 } } peers { permanent_uuid: "5ee313ff56e741b8a374762dbbbba570" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 45777 } } peers { permanent_uuid: "40feb304df974a3c989b1a15725c3503" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 43969 } }
I20260923 15:01:35.325301 14662 raft_consensus.cc:374] T fe1f1f411a084ed9bb043b0f8c446cba P 40feb304df974a3c989b1a15725c3503 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:01:35.325323 14662 raft_consensus.cc:729] T fe1f1f411a084ed9bb043b0f8c446cba P 40feb304df974a3c989b1a15725c3503 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 40feb304df974a3c989b1a15725c3503, State: Initialized, Role: FOLLOWER
I20260923 15:01:35.325326 14663 raft_consensus.cc:374] T fe1f1f411a084ed9bb043b0f8c446cba P 5ee313ff56e741b8a374762dbbbba570 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:01:35.325367 14663 raft_consensus.cc:729] T fe1f1f411a084ed9bb043b0f8c446cba P 5ee313ff56e741b8a374762dbbbba570 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5ee313ff56e741b8a374762dbbbba570, State: Initialized, Role: FOLLOWER
I20260923 15:01:35.325428 14662 consensus_queue.cc:260] T fe1f1f411a084ed9bb043b0f8c446cba P 40feb304df974a3c989b1a15725c3503 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c9724ac708540a0924d6b26b3dba6c6" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 33671 } } peers { permanent_uuid: "5ee313ff56e741b8a374762dbbbba570" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 45777 } } peers { permanent_uuid: "40feb304df974a3c989b1a15725c3503" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 43969 } }
I20260923 15:01:35.325560 14663 consensus_queue.cc:260] T fe1f1f411a084ed9bb043b0f8c446cba P 5ee313ff56e741b8a374762dbbbba570 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c9724ac708540a0924d6b26b3dba6c6" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 33671 } } peers { permanent_uuid: "5ee313ff56e741b8a374762dbbbba570" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 45777 } } peers { permanent_uuid: "40feb304df974a3c989b1a15725c3503" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 43969 } }
I20260923 15:01:35.325726 14662 ts_tablet_manager.cc:1428] T fe1f1f411a084ed9bb043b0f8c446cba P 40feb304df974a3c989b1a15725c3503: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260923 15:01:35.325865 14644 heartbeater.cc:503] Master 127.13.75.190:44465 was elected leader, sending a full tablet report...
I20260923 15:01:35.326131 14663 ts_tablet_manager.cc:1428] T fe1f1f411a084ed9bb043b0f8c446cba P 5ee313ff56e741b8a374762dbbbba570: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260923 15:01:35.326157 14510 heartbeater.cc:503] Master 127.13.75.190:44465 was elected leader, sending a full tablet report...
W20260923 15:01:35.327327 14511 tablet.cc:2369] T fe1f1f411a084ed9bb043b0f8c446cba P 5ee313ff56e741b8a374762dbbbba570: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260923 15:01:35.415967 14377 tablet.cc:2369] T fe1f1f411a084ed9bb043b0f8c446cba P 6c9724ac708540a0924d6b26b3dba6c6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 15:01:35.436857 14668 raft_consensus.cc:482] T fe1f1f411a084ed9bb043b0f8c446cba P 40feb304df974a3c989b1a15725c3503 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 15:01:35.437009 14668 raft_consensus.cc:504] T fe1f1f411a084ed9bb043b0f8c446cba P 40feb304df974a3c989b1a15725c3503 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c9724ac708540a0924d6b26b3dba6c6" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 33671 } } peers { permanent_uuid: "5ee313ff56e741b8a374762dbbbba570" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 45777 } } peers { permanent_uuid: "40feb304df974a3c989b1a15725c3503" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 43969 } }
I20260923 15:01:35.437400 14668 leader_election.cc:288] T fe1f1f411a084ed9bb043b0f8c446cba P 40feb304df974a3c989b1a15725c3503 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6c9724ac708540a0924d6b26b3dba6c6 (127.13.75.129:33671), 5ee313ff56e741b8a374762dbbbba570 (127.13.75.130:45777)
I20260923 15:01:35.441172 14658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:35.437682 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.131:55419 (local address 127.13.75.129:33671)
0923 15:01:35.437746 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0923 15:01:35.437748 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:01:35.437782 (+ 34us) server_negotiation.cc:407] Connection header received
0923 15:01:35.437890 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:01:35.437891 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:01:35.437902 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:01:35.437958 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:01:35.438215 (+ 257us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:35.438784 (+ 569us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:35.440240 (+ 1456us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:35.440782 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:35.440862 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:01:35.440870 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0923 15:01:35.441056 (+ 186us) server_negotiation.cc:299] Negotiation successful
0923 15:01:35.441090 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260923 15:01:35.441175 14646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:35.437618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.129:33671 (local address 127.13.75.131:55419)
0923 15:01:35.437697 (+ 79us) negotiation.cc:107] Waiting for socket to connect
0923 15:01:35.437724 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0923 15:01:35.437804 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:01:35.437993 (+ 189us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:01:35.437994 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:01:35.437998 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:01:35.438109 (+ 111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:01:35.438111 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:01:35.438865 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:01:35.438866 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:01:35.440146 (+ 1280us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:01:35.440149 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:01:35.440866 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:01:35.440867 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:01:35.440944 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:01:35.440948 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 15:01:35.441026 (+ 78us) client_negotiation.cc:240] Negotiation successful
0923 15:01:35.441072 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52,"mutex_wait_us":36}
I20260923 15:01:35.441287 14660 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:35.437822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.131:43393 (local address 127.13.75.130:45777)
0923 15:01:35.437871 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0923 15:01:35.437873 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:01:35.438004 (+ 131us) server_negotiation.cc:407] Connection header received
0923 15:01:35.438086 (+ 82us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:01:35.438087 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:01:35.438098 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:01:35.438166 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:01:35.438386 (+ 220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:35.439048 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:35.440353 (+ 1305us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:35.440925 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:35.440986 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:01:35.440992 (+ 6us) server_negotiation.cc:1035] Waiting for connection context
0923 15:01:35.441164 (+ 172us) server_negotiation.cc:299] Negotiation successful
0923 15:01:35.441211 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260923 15:01:35.441542 14670 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:35.437752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.130:45777 (local address 127.13.75.131:43393)
0923 15:01:35.437936 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0923 15:01:35.437945 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 15:01:35.438017 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:01:35.438215 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:01:35.438216 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:01:35.438218 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:01:35.438291 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:01:35.438293 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:01:35.439128 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:01:35.439128 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:01:35.440281 (+ 1153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:01:35.440283 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:01:35.441014 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:01:35.441015 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:01:35.441094 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:01:35.441098 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 15:01:35.441151 (+ 53us) client_negotiation.cc:240] Negotiation successful
0923 15:01:35.441179 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":113,"threads_started":1}
I20260923 15:01:35.441558 14330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe1f1f411a084ed9bb043b0f8c446cba" candidate_uuid: "40feb304df974a3c989b1a15725c3503" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c9724ac708540a0924d6b26b3dba6c6" is_pre_election: true
I20260923 15:01:35.441725 14330 raft_consensus.cc:2454] T fe1f1f411a084ed9bb043b0f8c446cba P 6c9724ac708540a0924d6b26b3dba6c6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 40feb304df974a3c989b1a15725c3503 in term 0.
I20260923 15:01:35.441874 14464 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe1f1f411a084ed9bb043b0f8c446cba" candidate_uuid: "40feb304df974a3c989b1a15725c3503" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ee313ff56e741b8a374762dbbbba570" is_pre_election: true
I20260923 15:01:35.442014 14464 raft_consensus.cc:2454] T fe1f1f411a084ed9bb043b0f8c446cba P 5ee313ff56e741b8a374762dbbbba570 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 40feb304df974a3c989b1a15725c3503 in term 0.
I20260923 15:01:35.441987 14532 leader_election.cc:302] T fe1f1f411a084ed9bb043b0f8c446cba P 40feb304df974a3c989b1a15725c3503 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 40feb304df974a3c989b1a15725c3503, 6c9724ac708540a0924d6b26b3dba6c6; no voters:
I20260923 15:01:35.442194 14668 raft_consensus.cc:2781] T fe1f1f411a084ed9bb043b0f8c446cba P 40feb304df974a3c989b1a15725c3503 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 15:01:35.442256 14668 raft_consensus.cc:482] T fe1f1f411a084ed9bb043b0f8c446cba P 40feb304df974a3c989b1a15725c3503 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 15:01:35.442273 14668 raft_consensus.cc:3037] T fe1f1f411a084ed9bb043b0f8c446cba P 40feb304df974a3c989b1a15725c3503 [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:01:35.443048 14668 raft_consensus.cc:504] T fe1f1f411a084ed9bb043b0f8c446cba P 40feb304df974a3c989b1a15725c3503 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c9724ac708540a0924d6b26b3dba6c6" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 33671 } } peers { permanent_uuid: "5ee313ff56e741b8a374762dbbbba570" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 45777 } } peers { permanent_uuid: "40feb304df974a3c989b1a15725c3503" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 43969 } }
I20260923 15:01:35.443181 14668 leader_election.cc:288] T fe1f1f411a084ed9bb043b0f8c446cba P 40feb304df974a3c989b1a15725c3503 [CANDIDATE]: Term 1 election: Requested vote from peers 6c9724ac708540a0924d6b26b3dba6c6 (127.13.75.129:33671), 5ee313ff56e741b8a374762dbbbba570 (127.13.75.130:45777)
I20260923 15:01:35.443322 14330 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe1f1f411a084ed9bb043b0f8c446cba" candidate_uuid: "40feb304df974a3c989b1a15725c3503" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6c9724ac708540a0924d6b26b3dba6c6"
I20260923 15:01:35.443397 14330 raft_consensus.cc:3037] T fe1f1f411a084ed9bb043b0f8c446cba P 6c9724ac708540a0924d6b26b3dba6c6 [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:01:35.443378 14464 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fe1f1f411a084ed9bb043b0f8c446cba" candidate_uuid: "40feb304df974a3c989b1a15725c3503" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5ee313ff56e741b8a374762dbbbba570"
I20260923 15:01:35.443478 14464 raft_consensus.cc:3037] T fe1f1f411a084ed9bb043b0f8c446cba P 5ee313ff56e741b8a374762dbbbba570 [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:01:35.444113 14330 raft_consensus.cc:2454] T fe1f1f411a084ed9bb043b0f8c446cba P 6c9724ac708540a0924d6b26b3dba6c6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 40feb304df974a3c989b1a15725c3503 in term 1.
I20260923 15:01:35.444276 14532 leader_election.cc:302] T fe1f1f411a084ed9bb043b0f8c446cba P 40feb304df974a3c989b1a15725c3503 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 40feb304df974a3c989b1a15725c3503, 6c9724ac708540a0924d6b26b3dba6c6; no voters:
I20260923 15:01:35.444312 14464 raft_consensus.cc:2454] T fe1f1f411a084ed9bb043b0f8c446cba P 5ee313ff56e741b8a374762dbbbba570 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 40feb304df974a3c989b1a15725c3503 in term 1.
I20260923 15:01:35.444371 14668 raft_consensus.cc:2781] T fe1f1f411a084ed9bb043b0f8c446cba P 40feb304df974a3c989b1a15725c3503 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 15:01:35.444538 14668 raft_consensus.cc:686] T fe1f1f411a084ed9bb043b0f8c446cba P 40feb304df974a3c989b1a15725c3503 [term 1 LEADER]: Becoming Leader. State: Replica: 40feb304df974a3c989b1a15725c3503, State: Running, Role: LEADER
I20260923 15:01:35.444669 14668 consensus_queue.cc:237] T fe1f1f411a084ed9bb043b0f8c446cba P 40feb304df974a3c989b1a15725c3503 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c9724ac708540a0924d6b26b3dba6c6" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 33671 } } peers { permanent_uuid: "5ee313ff56e741b8a374762dbbbba570" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 45777 } } peers { permanent_uuid: "40feb304df974a3c989b1a15725c3503" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 43969 } }
I20260923 15:01:35.445506 14185 catalog_manager.cc:5495] T fe1f1f411a084ed9bb043b0f8c446cba P 40feb304df974a3c989b1a15725c3503 reported cstate change: term changed from 0 to 1, leader changed from <none> to 40feb304df974a3c989b1a15725c3503 (127.13.75.131). New cstate: current_term: 1 leader_uuid: "40feb304df974a3c989b1a15725c3503" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6c9724ac708540a0924d6b26b3dba6c6" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 33671 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5ee313ff56e741b8a374762dbbbba570" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 45777 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "40feb304df974a3c989b1a15725c3503" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 43969 } health_report { overall_health: HEALTHY } } }
Sep 23 15:01:35 dist-test-slave-cnb0 krb5kdc[14140](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175694, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.75.129@KRBTEST.COM
I20260923 15:01:35.463600 14658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:35.455885 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41156 (local address 127.13.75.129:33671)
0923 15:01:35.455947 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0923 15:01:35.455948 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 15:01:35.455956 (+ 8us) server_negotiation.cc:407] Connection header received
0923 15:01:35.455993 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:01:35.455993 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:01:35.456003 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:01:35.456060 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:01:35.456604 (+ 544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:35.457092 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:35.457782 (+ 690us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:35.458005 (+ 223us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:35.458073 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:01:35.460561 (+ 2488us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:01:35.460575 (+ 14us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:01:35.460575 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:01:35.460583 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:01:35.462242 (+ 1659us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:01:35.462635 (+ 393us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:01:35.462640 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:01:35.462640 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:01:35.462721 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:01:35.462997 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:01:35.462998 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:01:35.462999 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:01:35.463171 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:01:35.463333 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0923 15:01:35.463380 (+ 47us) server_negotiation.cc:299] Negotiation successful
0923 15:01:35.463473 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
Sep 23 15:01:35 dist-test-slave-cnb0 krb5kdc[14140](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175695, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 23 15:01:35 dist-test-slave-cnb0 krb5kdc[14140](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175695, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.13.75.129@KRBTEST.COM
I20260923 15:01:35.487025 14658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:35.479721 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41168 (local address 127.13.75.129:33671)
0923 15:01:35.479793 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0923 15:01:35.479795 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:01:35.479842 (+ 47us) server_negotiation.cc:407] Connection header received
0923 15:01:35.479890 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:01:35.479891 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:01:35.479902 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:01:35.479957 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:01:35.480505 (+ 548us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:35.480992 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:35.481726 (+ 734us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:35.481930 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:35.481994 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:01:35.484293 (+ 2299us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:01:35.484294 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:01:35.484295 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:01:35.484302 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:01:35.485733 (+ 1431us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:01:35.486066 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:01:35.486067 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:01:35.486068 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:01:35.486133 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:01:35.486353 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:01:35.486354 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:01:35.486354 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:01:35.486473 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:01:35.486559 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0923 15:01:35.486747 (+ 188us) server_negotiation.cc:299] Negotiation successful
0923 15:01:35.486818 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
Sep 23 15:01:35 dist-test-slave-cnb0 krb5kdc[14140](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175695, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 23 15:01:35 dist-test-slave-cnb0 krb5kdc[14140](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175695, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.75.129@KRBTEST.COM
I20260923 15:01:35.510211 14658 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:35.503024 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41184 (local address 127.13.75.129:33671)
0923 15:01:35.503112 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0923 15:01:35.503114 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:01:35.503207 (+ 93us) server_negotiation.cc:407] Connection header received
0923 15:01:35.503238 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:01:35.503239 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:01:35.503250 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:01:35.503323 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:01:35.503951 (+ 628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:35.504499 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:35.505227 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:35.505447 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:35.505501 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:01:35.507822 (+ 2321us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:01:35.507824 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:01:35.507824 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:01:35.507831 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:01:35.509137 (+ 1306us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:01:35.509467 (+ 330us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:01:35.509468 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:01:35.509468 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:01:35.509529 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:01:35.509765 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:01:35.509766 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:01:35.509766 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:01:35.509863 (+ 97us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:01:35.509937 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0923 15:01:35.510047 (+ 110us) server_negotiation.cc:299] Negotiation successful
0923 15:01:35.510109 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
W20260923 15:01:35.510794 14290 tablet_service.cc:2908] Rejecting scan request for tablet fe1f1f411a084ed9bb043b0f8c446cba: Uninitialized: clean time has not yet been initialized
I20260923 15:01:35.511749 13614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKC4soy/build/release/bin/kudu with pid 14247
I20260923 15:01:35.517373 13614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKC4soy/build/release/bin/kudu with pid 14381
I20260923 15:01:35.522071 13614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKC4soy/build/release/bin/kudu with pid 14515
I20260923 15:01:35.527348 13614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKC4soy/build/release/bin/kudu with pid 14153
2026-09-23T15:01:35Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizationOnChecksum (2109 ms)
[ RUN ] SecurityITest.SmokeTestAsAuthorizedUser
Loading random data
Initializing database '/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175691491353-13614-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 15:01:35 dist-test-slave-cnb0 krb5kdc[14685](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 15:01:35 dist-test-slave-cnb0 krb5kdc[14685](info): set up 2 sockets
Sep 23 15:01:35 dist-test-slave-cnb0 krb5kdc[14685](info): commencing operation
krb5kdc: starting...
W20260923 15:01:36.571931 13614 mini_kdc.cc:121] Time spent starting KDC: real 1.023s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175691491353-13614-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175691491353-13614-0/krb5kdc/test-user.keytab.
Sep 23 15:01:36 dist-test-slave-cnb0 krb5kdc[14685](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175696, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T15:01:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T15:01:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.75.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.75.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.75.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.190.keytab.
Entry for principal kudu/127.13.75.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.190.keytab.
I20260923 15:01:36.691941 13614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKC4soy/build/release/bin/kudu
/tmp/dist-test-taskKC4soy/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175691491353-13614-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175691491353-13614-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175691491353-13614-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175691491353-13614-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.75.190:35253
--webserver_interface=127.13.75.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.190.keytab
--principal=kudu/127.13.75.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.75.148:36005
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.75.190:35253
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175691491353-13614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175691491353-13614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175691491353-13614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:01:36.773068 14698 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:01:36.773327 14698 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:01:36.773366 14698 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:01:36.774549 14698 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 15:01:36.774613 14698 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260923 15:01:36.774637 14698 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:01:36.774662 14698 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:01:36.774690 14698 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 15:01:36.774719 14698 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 15:01:36.776253 14698 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.75.148:36005
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175691491353-13614-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175691491353-13614-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.75.190:35253
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.190.keytab
--principal=kudu/127.13.75.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.75.190:35253
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175691491353-13614-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.75.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14698
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175691491353-13614-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:01:36.776551 14698 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:01:36.776782 14698 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:01:36.781670 14706 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:01:36.781697 14703 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:01:36.781716 14704 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:01:36.781906 14698 server_base.cc:1034] running on GCE node
I20260923 15:01:36.782258 14698 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:01:36.782603 14698 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:01:36.783790 14698 hybrid_clock.cc:648] HybridClock initialized: now 1790175696783778 us; error 36 us; skew 500 ppm
Sep 23 15:01:36 dist-test-slave-cnb0 krb5kdc[14685](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175696, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:01:36.792130 14698 init.cc:377] Logged in from keytab as kudu/127.13.75.190@KRBTEST.COM (short username kudu)
I20260923 15:01:36.792645 14698 webserver.cc:460] Webserver started at http://127.13.75.190:43809/ using document root <none> and password file <none>
I20260923 15:01:36.792908 14698 fs_manager.cc:359] Metadata directory not provided
I20260923 15:01:36.792980 14698 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:01:36.793135 14698 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:01:36.794317 14698 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175691491353-13614-0/minicluster-data/master-0/data/instance:
uuid: "61f4472f8f444f179d09ce5ce961925c"
format_stamp: "Formatted at 2026-09-23 15:01:36 on dist-test-slave-cnb0"
server_key: "f1121fa977275d580cb39358bf905c4e"
server_key_iv: "fae460577a0d767a8567e2fc6a218a7e"
server_key_version: "encryptionkey@0"
I20260923 15:01:36.794693 14698 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175691491353-13614-0/minicluster-data/master-0/wal/instance:
uuid: "61f4472f8f444f179d09ce5ce961925c"
format_stamp: "Formatted at 2026-09-23 15:01:36 on dist-test-slave-cnb0"
server_key: "f1121fa977275d580cb39358bf905c4e"
server_key_iv: "fae460577a0d767a8567e2fc6a218a7e"
server_key_version: "encryptionkey@0"
I20260923 15:01:36.796728 14698 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.001s
I20260923 15:01:36.797636 14713 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:01:36.797894 14698 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:01:36.798010 14698 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175691491353-13614-0/minicluster-data/master-0/data,/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175691491353-13614-0/minicluster-data/master-0/wal
uuid: "61f4472f8f444f179d09ce5ce961925c"
format_stamp: "Formatted at 2026-09-23 15:01:36 on dist-test-slave-cnb0"
server_key: "f1121fa977275d580cb39358bf905c4e"
server_key_iv: "fae460577a0d767a8567e2fc6a218a7e"
server_key_version: "encryptionkey@0"
I20260923 15:01:36.798094 14698 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175691491353-13614-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175691491353-13614-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175691491353-13614-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:01:36.821914 14698 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:01:36.824239 14698 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:01:36.824416 14698 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:01:36.829157 14698 rpc_server.cc:307] RPC server started. Bound to: 127.13.75.190:35253
I20260923 15:01:36.829269 14775 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.75.190:35253 every 8 connection(s)
I20260923 15:01:36.829581 14698 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175691491353-13614-0/minicluster-data/master-0/data/info.pb
I20260923 15:01:36.830286 14776 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:01:36.832677 14776 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 61f4472f8f444f179d09ce5ce961925c: Bootstrap starting.
I20260923 15:01:36.833303 14776 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 61f4472f8f444f179d09ce5ce961925c: Neither blocks nor log segments found. Creating new log.
I20260923 15:01:36.833608 14776 log.cc:824] T 00000000000000000000000000000000 P 61f4472f8f444f179d09ce5ce961925c: Log is configured to *not* fsync() on all Append() calls
I20260923 15:01:36.834353 14776 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 61f4472f8f444f179d09ce5ce961925c: No bootstrap required, opened a new log
I20260923 15:01:36.836611 14776 raft_consensus.cc:348] T 00000000000000000000000000000000 P 61f4472f8f444f179d09ce5ce961925c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61f4472f8f444f179d09ce5ce961925c" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 35253 } }
I20260923 15:01:36.836771 14776 raft_consensus.cc:374] T 00000000000000000000000000000000 P 61f4472f8f444f179d09ce5ce961925c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:01:36.836794 14776 raft_consensus.cc:729] T 00000000000000000000000000000000 P 61f4472f8f444f179d09ce5ce961925c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 61f4472f8f444f179d09ce5ce961925c, State: Initialized, Role: FOLLOWER
I20260923 15:01:36.836953 14776 consensus_queue.cc:260] T 00000000000000000000000000000000 P 61f4472f8f444f179d09ce5ce961925c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61f4472f8f444f179d09ce5ce961925c" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 35253 } }
I20260923 15:01:36.837025 14776 raft_consensus.cc:388] T 00000000000000000000000000000000 P 61f4472f8f444f179d09ce5ce961925c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 15:01:36.837056 14776 raft_consensus.cc:482] T 00000000000000000000000000000000 P 61f4472f8f444f179d09ce5ce961925c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 15:01:36.837085 14776 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 61f4472f8f444f179d09ce5ce961925c [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:01:36.837267 13614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKC4soy/build/release/bin/kudu as pid 14698
I20260923 15:01:36.837370 13614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175691491353-13614-0/minicluster-data/master-0/wal/instance
I20260923 15:01:36.837531 13614 external_mini_cluster.cc:1428] Setting key db3835835d0d77722699b97295ba7664
I20260923 15:01:36.837800 14776 raft_consensus.cc:504] T 00000000000000000000000000000000 P 61f4472f8f444f179d09ce5ce961925c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61f4472f8f444f179d09ce5ce961925c" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 35253 } }
I20260923 15:01:36.837929 14776 leader_election.cc:302] T 00000000000000000000000000000000 P 61f4472f8f444f179d09ce5ce961925c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 61f4472f8f444f179d09ce5ce961925c; no voters:
I20260923 15:01:36.838194 14776 leader_election.cc:288] T 00000000000000000000000000000000 P 61f4472f8f444f179d09ce5ce961925c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 15:01:36.838225 14780 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 61f4472f8f444f179d09ce5ce961925c [term 1 FOLLOWER]: Leader election won for term 1
I20260923 15:01:36.838395 14780 raft_consensus.cc:686] T 00000000000000000000000000000000 P 61f4472f8f444f179d09ce5ce961925c [term 1 LEADER]: Becoming Leader. State: Replica: 61f4472f8f444f179d09ce5ce961925c, State: Running, Role: LEADER
I20260923 15:01:36.838574 14776 sys_catalog.cc:564] T 00000000000000000000000000000000 P 61f4472f8f444f179d09ce5ce961925c [sys.catalog]: configured and running, proceeding with master startup.
I20260923 15:01:36.838546 14780 consensus_queue.cc:237] T 00000000000000000000000000000000 P 61f4472f8f444f179d09ce5ce961925c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61f4472f8f444f179d09ce5ce961925c" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 35253 } }
I20260923 15:01:36.839061 14782 sys_catalog.cc:455] T 00000000000000000000000000000000 P 61f4472f8f444f179d09ce5ce961925c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 61f4472f8f444f179d09ce5ce961925c. Latest consensus state: current_term: 1 leader_uuid: "61f4472f8f444f179d09ce5ce961925c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61f4472f8f444f179d09ce5ce961925c" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 35253 } } }
I20260923 15:01:36.839151 14782 sys_catalog.cc:458] T 00000000000000000000000000000000 P 61f4472f8f444f179d09ce5ce961925c [sys.catalog]: This master's current role is: LEADER
I20260923 15:01:36.839169 14783 sys_catalog.cc:455] T 00000000000000000000000000000000 P 61f4472f8f444f179d09ce5ce961925c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "61f4472f8f444f179d09ce5ce961925c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "61f4472f8f444f179d09ce5ce961925c" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 35253 } } }
I20260923 15:01:36.839259 14783 sys_catalog.cc:458] T 00000000000000000000000000000000 P 61f4472f8f444f179d09ce5ce961925c [sys.catalog]: This master's current role is: LEADER
I20260923 15:01:36.840251 14788 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 15:01:36.840878 14788 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 15:01:36.842984 14788 catalog_manager.cc:1347] Generated new cluster ID: 04d34b74f3e04fa491af1423324cf4ab
I20260923 15:01:36.843042 14788 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 23 15:01:36 dist-test-slave-cnb0 krb5kdc[14685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175696, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.75.190@KRBTEST.COM
I20260923 15:01:36.848572 14781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:36.838062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45292 (local address 127.13.75.190:35253)
0923 15:01:36.838277 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0923 15:01:36.838288 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0923 15:01:36.838319 (+ 31us) server_negotiation.cc:407] Connection header received
0923 15:01:36.838506 (+ 187us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:01:36.838514 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:01:36.838578 (+ 64us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:01:36.839719 (+ 1141us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:01:36.839837 (+ 118us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:36.841244 (+ 1407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:36.842159 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:36.842733 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:36.842862 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:01:36.845228 (+ 2366us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:01:36.845232 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:01:36.845232 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:01:36.845246 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:01:36.846985 (+ 1739us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:01:36.847343 (+ 358us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:01:36.847346 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:01:36.847347 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:01:36.847465 (+ 118us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:01:36.847681 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:01:36.847682 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:01:36.847683 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:01:36.847929 (+ 246us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:01:36.848104 (+ 175us) server_negotiation.cc:1035] Waiting for connection context
0923 15:01:36.848230 (+ 126us) server_negotiation.cc:299] Negotiation successful
0923 15:01:36.848385 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":102,"threads_started":1}
I20260923 15:01:36.855077 14788 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 15:01:36.855620 14788 catalog_manager.cc:1504] Loading token signing keys...
I20260923 15:01:36.860486 14788 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 61f4472f8f444f179d09ce5ce961925c: Generated new TSK 0
I20260923 15:01:36.860683 14788 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.75.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.75.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.75.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.129.keytab.
Entry for principal kudu/127.13.75.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.129.keytab.
I20260923 15:01:36.894035 13614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKC4soy/build/release/bin/kudu
/tmp/dist-test-taskKC4soy/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175691491353-13614-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175691491353-13614-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175691491353-13614-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175691491353-13614-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.75.129:0
--local_ip_for_outbound_sockets=127.13.75.129
--webserver_interface=127.13.75.129
--webserver_port=0
--tserver_master_addrs=127.13.75.190:35253
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.129.keytab
--principal=kudu/127.13.75.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.75.148:36005
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175691491353-13614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175691491353-13614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175691491353-13614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:01:36.999876 14802 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:01:37.000109 14802 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:01:37.000137 14802 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:01:37.001452 14802 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:01:37.001506 14802 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:01:37.001529 14802 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260923 15:01:37.001549 14802 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.75.129
I20260923 15:01:37.003743 14802 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.75.148:36005
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175691491353-13614-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175691491353-13614-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.129.keytab
--principal=kudu/127.13.75.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.75.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175691491353-13614-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.75.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.13.75.190:35253
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14802
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.75.129
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175691491353-13614-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:01:37.004068 14802 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:01:37.004357 14802 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:01:37.010382 14808 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:01:37.010485 14802 server_base.cc:1034] running on GCE node
W20260923 15:01:37.010392 14807 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:01:37.010392 14810 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:01:37.010846 14802 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:01:37.011162 14802 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:01:37.012333 14802 hybrid_clock.cc:648] HybridClock initialized: now 1790175697012326 us; error 35 us; skew 500 ppm
Sep 23 15:01:37 dist-test-slave-cnb0 krb5kdc[14685](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175697, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:01:37.015117 14802 init.cc:377] Logged in from keytab as kudu/127.13.75.129@KRBTEST.COM (short username kudu)
I20260923 15:01:37.015599 14802 webserver.cc:460] Webserver started at http://127.13.75.129:43423/ using document root <none> and password file <none>
I20260923 15:01:37.015833 14802 fs_manager.cc:359] Metadata directory not provided
I20260923 15:01:37.015898 14802 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:01:37.016048 14802 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:01:37.017141 14802 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175691491353-13614-0/minicluster-data/ts-0/data/instance:
uuid: "1cebda21ec684fcabfa5beaef4edda08"
format_stamp: "Formatted at 2026-09-23 15:01:37 on dist-test-slave-cnb0"
server_key: "f783f0e628a9a40f473f04de30d9a936"
server_key_iv: "8d4e55cac9ddbc093ce9ed6b78806d8f"
server_key_version: "encryptionkey@0"
I20260923 15:01:37.017499 14802 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175691491353-13614-0/minicluster-data/ts-0/wal/instance:
uuid: "1cebda21ec684fcabfa5beaef4edda08"
format_stamp: "Formatted at 2026-09-23 15:01:37 on dist-test-slave-cnb0"
server_key: "f783f0e628a9a40f473f04de30d9a936"
server_key_iv: "8d4e55cac9ddbc093ce9ed6b78806d8f"
server_key_version: "encryptionkey@0"
I20260923 15:01:37.018851 14802 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260923 15:01:37.019694 14817 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:01:37.019877 14802 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:01:37.019975 14802 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175691491353-13614-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175691491353-13614-0/minicluster-data/ts-0/wal
uuid: "1cebda21ec684fcabfa5beaef4edda08"
format_stamp: "Formatted at 2026-09-23 15:01:37 on dist-test-slave-cnb0"
server_key: "f783f0e628a9a40f473f04de30d9a936"
server_key_iv: "8d4e55cac9ddbc093ce9ed6b78806d8f"
server_key_version: "encryptionkey@0"
I20260923 15:01:37.020056 14802 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175691491353-13614-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175691491353-13614-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175691491353-13614-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:01:37.033783 14802 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:01:37.036123 14802 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:01:37.036306 14802 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:01:37.036880 14802 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 15:01:37.036945 14802 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 15:01:37.036981 14802 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 15:01:37.037009 14802 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 15:01:37.043814 14802 rpc_server.cc:307] RPC server started. Bound to: 127.13.75.129:43089
I20260923 15:01:37.043865 14932 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.75.129:43089 every 8 connection(s)
Sep 23 15:01:37 dist-test-slave-cnb0 krb5kdc[14685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175697, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.129@KRBTEST.COM for kudu/127.13.75.190@KRBTEST.COM
I20260923 15:01:37.044250 14802 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175691491353-13614-0/minicluster-data/ts-0/data/info.pb
I20260923 15:01:37.047672 14781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:37.037418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.129:41423 (local address 127.13.75.190:35253)
0923 15:01:37.037499 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0923 15:01:37.037502 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:01:37.037660 (+ 158us) server_negotiation.cc:407] Connection header received
0923 15:01:37.038128 (+ 468us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:01:37.038130 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:01:37.038148 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:01:37.038228 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:01:37.039296 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:37.040146 (+ 850us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:37.041756 (+ 1610us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:37.042015 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:37.042110 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:01:37.044857 (+ 2747us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:01:37.044858 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:01:37.044859 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:01:37.044870 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:01:37.046193 (+ 1323us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:01:37.046617 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:01:37.046619 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:01:37.046619 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:01:37.046685 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:01:37.046975 (+ 290us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:01:37.046977 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:01:37.046977 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:01:37.047110 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:01:37.047190 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0923 15:01:37.047471 (+ 281us) server_negotiation.cc:299] Negotiation successful
0923 15:01:37.047576 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260923 15:01:37.047858 14826 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:37.037337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:35253 (local address 127.13.75.129:41423)
0923 15:01:37.037515 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0923 15:01:37.037564 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0923 15:01:37.038028 (+ 464us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:01:37.038319 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:01:37.038323 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:01:37.038731 (+ 408us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:01:37.039184 (+ 453us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:01:37.039191 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:01:37.040279 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:01:37.040281 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:01:37.041665 (+ 1384us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:01:37.041669 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:01:37.042120 (+ 451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:01:37.042121 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:01:37.042279 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:01:37.042875 (+ 596us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:01:37.042921 (+ 46us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:01:37.044752 (+ 1831us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:01:37.046292 (+ 1540us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:01:37.046299 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:01:37.046301 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:01:37.046539 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:01:37.046750 (+ 211us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:01:37.046751 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:01:37.046752 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:01:37.046870 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:01:37.047179 (+ 309us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:01:37.047186 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:01:37.047334 (+ 148us) client_negotiation.cc:769] Sending connection context
0923 15:01:37.047436 (+ 102us) client_negotiation.cc:240] Negotiation successful
0923 15:01:37.047612 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":84,"threads_started":1}
I20260923 15:01:37.048380 14933 heartbeater.cc:347] Connected to a master server at 127.13.75.190:35253
I20260923 15:01:37.048470 14933 heartbeater.cc:464] Registering TS with master...
I20260923 15:01:37.048660 14933 heartbeater.cc:511] Master 127.13.75.190:35253 requested a full tablet report, sending...
I20260923 15:01:37.049181 14730 ts_manager.cc:194] Registered new tserver with Master: 1cebda21ec684fcabfa5beaef4edda08 (127.13.75.129:43089)
I20260923 15:01:37.049955 13614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKC4soy/build/release/bin/kudu as pid 14802
I20260923 15:01:37.050113 13614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175691491353-13614-0/minicluster-data/ts-0/wal/instance
I20260923 15:01:37.050112 14730 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.75.129@KRBTEST.COM'} at 127.13.75.129:41423
I20260923 15:01:37.050309 13614 external_mini_cluster.cc:1428] Setting key dda9dacc02838e256d152ef41af3831c
I20260923 15:01:37.055739 14781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:37.049596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.129:48763 (local address 127.13.75.190:35253)
0923 15:01:37.049634 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0923 15:01:37.049636 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:01:37.049781 (+ 145us) server_negotiation.cc:407] Connection header received
0923 15:01:37.049842 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:01:37.049843 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:01:37.049853 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:01:37.049907 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:01:37.050374 (+ 467us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:37.051084 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:37.051719 (+ 635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:37.052063 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:37.052135 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:01:37.053174 (+ 1039us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:01:37.053175 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:01:37.053175 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:01:37.053181 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:01:37.054561 (+ 1380us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:01:37.054999 (+ 438us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:01:37.055000 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:01:37.055001 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:01:37.055052 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:01:37.055255 (+ 203us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:01:37.055255 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:01:37.055255 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:01:37.055355 (+ 100us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:01:37.055416 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0923 15:01:37.055603 (+ 187us) server_negotiation.cc:299] Negotiation successful
0923 15:01:37.055676 (+ 73us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260923 15:01:37.055852 14939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:37.049530 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:35253 (local address 127.13.75.129:48763)
0923 15:01:37.049698 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0923 15:01:37.049708 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 15:01:37.049779 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:01:37.049923 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:01:37.049924 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:01:37.050190 (+ 266us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:01:37.050269 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:01:37.050271 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:01:37.051156 (+ 885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:01:37.051156 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:01:37.051642 (+ 486us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:01:37.051643 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:01:37.052140 (+ 497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:01:37.052141 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:01:37.052226 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:01:37.052665 (+ 439us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:01:37.052671 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:01:37.053083 (+ 412us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:01:37.054655 (+ 1572us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:01:37.054656 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:01:37.054657 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:01:37.054923 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:01:37.055118 (+ 195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:01:37.055119 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:01:37.055120 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:01:37.055190 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:01:37.055426 (+ 236us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:01:37.055427 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:01:37.055483 (+ 56us) client_negotiation.cc:769] Sending connection context
0923 15:01:37.055576 (+ 93us) client_negotiation.cc:240] Negotiation successful
0923 15:01:37.055700 (+ 124us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":69,"threads_started":1}
WARNING: no policy specified for kudu/127.13.75.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.75.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.75.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.130.keytab.
Entry for principal kudu/127.13.75.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.130.keytab.
I20260923 15:01:37.077862 13614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKC4soy/build/release/bin/kudu
/tmp/dist-test-taskKC4soy/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175691491353-13614-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175691491353-13614-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175691491353-13614-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175691491353-13614-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.75.130:0
--local_ip_for_outbound_sockets=127.13.75.130
--webserver_interface=127.13.75.130
--webserver_port=0
--tserver_master_addrs=127.13.75.190:35253
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.130.keytab
--principal=kudu/127.13.75.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.75.148:36005
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175691491353-13614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175691491353-13614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175691491353-13614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:01:37.162320 14942 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:01:37.162535 14942 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:01:37.162560 14942 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:01:37.163697 14942 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:01:37.163736 14942 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:01:37.163756 14942 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260923 15:01:37.163774 14942 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.75.130
I20260923 15:01:37.165194 14942 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.75.148:36005
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175691491353-13614-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175691491353-13614-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.130.keytab
--principal=kudu/127.13.75.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.75.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175691491353-13614-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.75.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.13.75.190:35253
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.14942
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.75.130
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175691491353-13614-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:01:37.165405 14942 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:01:37.165596 14942 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:01:37.170583 14947 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:01:37.170557 14950 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:01:37.170730 14942 server_base.cc:1034] running on GCE node
W20260923 15:01:37.170612 14948 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:01:37.171098 14942 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:01:37.171350 14942 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:01:37.172529 14942 hybrid_clock.cc:648] HybridClock initialized: now 1790175697172517 us; error 42 us; skew 500 ppm
Sep 23 15:01:37 dist-test-slave-cnb0 krb5kdc[14685](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175697, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:01:37.175123 14942 init.cc:377] Logged in from keytab as kudu/127.13.75.130@KRBTEST.COM (short username kudu)
I20260923 15:01:37.175580 14942 webserver.cc:460] Webserver started at http://127.13.75.130:38567/ using document root <none> and password file <none>
I20260923 15:01:37.175832 14942 fs_manager.cc:359] Metadata directory not provided
I20260923 15:01:37.175880 14942 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:01:37.176033 14942 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:01:37.177103 14942 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175691491353-13614-0/minicluster-data/ts-1/data/instance:
uuid: "d046a0f9cc58427986a4fa16815e54e6"
format_stamp: "Formatted at 2026-09-23 15:01:37 on dist-test-slave-cnb0"
server_key: "cbe60157867fd474aa8790e98929d492"
server_key_iv: "bb6d084b7cac1e8757d3766107d342bb"
server_key_version: "encryptionkey@0"
I20260923 15:01:37.177477 14942 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175691491353-13614-0/minicluster-data/ts-1/wal/instance:
uuid: "d046a0f9cc58427986a4fa16815e54e6"
format_stamp: "Formatted at 2026-09-23 15:01:37 on dist-test-slave-cnb0"
server_key: "cbe60157867fd474aa8790e98929d492"
server_key_iv: "bb6d084b7cac1e8757d3766107d342bb"
server_key_version: "encryptionkey@0"
I20260923 15:01:37.178944 14942 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 15:01:37.179683 14957 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:01:37.179845 14942 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 15:01:37.179940 14942 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175691491353-13614-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175691491353-13614-0/minicluster-data/ts-1/wal
uuid: "d046a0f9cc58427986a4fa16815e54e6"
format_stamp: "Formatted at 2026-09-23 15:01:37 on dist-test-slave-cnb0"
server_key: "cbe60157867fd474aa8790e98929d492"
server_key_iv: "bb6d084b7cac1e8757d3766107d342bb"
server_key_version: "encryptionkey@0"
I20260923 15:01:37.180018 14942 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175691491353-13614-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175691491353-13614-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175691491353-13614-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:01:37.190572 14942 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:01:37.192449 14942 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:01:37.192611 14942 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:01:37.193192 14942 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 15:01:37.193255 14942 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 15:01:37.193290 14942 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 15:01:37.193317 14942 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 15:01:37.200232 14942 rpc_server.cc:307] RPC server started. Bound to: 127.13.75.130:35027
I20260923 15:01:37.200309 15072 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.75.130:35027 every 8 connection(s)
Sep 23 15:01:37 dist-test-slave-cnb0 krb5kdc[14685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175697, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.130@KRBTEST.COM for kudu/127.13.75.190@KRBTEST.COM
I20260923 15:01:37.200716 14942 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175691491353-13614-0/minicluster-data/ts-1/data/info.pb
I20260923 15:01:37.203126 13614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKC4soy/build/release/bin/kudu as pid 14942
I20260923 15:01:37.203229 13614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175691491353-13614-0/minicluster-data/ts-1/wal/instance
I20260923 15:01:37.203425 13614 external_mini_cluster.cc:1428] Setting key e1cc2b7dac55fe5e80adbac3a303feb8
I20260923 15:01:37.203846 14781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:37.193645 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.130:38195 (local address 127.13.75.190:35253)
0923 15:01:37.193699 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0923 15:01:37.193702 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:01:37.193914 (+ 212us) server_negotiation.cc:407] Connection header received
0923 15:01:37.194395 (+ 481us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:01:37.194397 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:01:37.194411 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:01:37.194490 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:01:37.195422 (+ 932us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:37.196274 (+ 852us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:37.197872 (+ 1598us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:37.198204 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:37.198282 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:01:37.201115 (+ 2833us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:01:37.201116 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:01:37.201117 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:01:37.201125 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:01:37.202484 (+ 1359us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:01:37.202859 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:01:37.202860 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:01:37.202860 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:01:37.202956 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:01:37.203192 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:01:37.203193 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:01:37.203193 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:01:37.203345 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:01:37.203426 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0923 15:01:37.203698 (+ 272us) server_negotiation.cc:299] Negotiation successful
0923 15:01:37.203783 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260923 15:01:37.204058 14966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:37.193620 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:35253 (local address 127.13.75.130:38195)
0923 15:01:37.193769 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0923 15:01:37.193809 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0923 15:01:37.194298 (+ 489us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:01:37.194576 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:01:37.194596 (+ 20us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:01:37.194974 (+ 378us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:01:37.195295 (+ 321us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:01:37.195302 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:01:37.196386 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:01:37.196387 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:01:37.197772 (+ 1385us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:01:37.197776 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:01:37.198301 (+ 525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:01:37.198302 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:01:37.198404 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:01:37.199036 (+ 632us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:01:37.199048 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:01:37.201020 (+ 1972us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:01:37.202570 (+ 1550us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:01:37.202576 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:01:37.202577 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:01:37.202792 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:01:37.203029 (+ 237us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:01:37.203030 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:01:37.203030 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:01:37.203129 (+ 99us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:01:37.203416 (+ 287us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:01:37.203420 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:01:37.203582 (+ 162us) client_negotiation.cc:769] Sending connection context
0923 15:01:37.203668 (+ 86us) client_negotiation.cc:240] Negotiation successful
0923 15:01:37.203839 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":82,"threads_started":1}
I20260923 15:01:37.204530 15073 heartbeater.cc:347] Connected to a master server at 127.13.75.190:35253
I20260923 15:01:37.204615 15073 heartbeater.cc:464] Registering TS with master...
I20260923 15:01:37.204839 15073 heartbeater.cc:511] Master 127.13.75.190:35253 requested a full tablet report, sending...
I20260923 15:01:37.205194 14730 ts_manager.cc:194] Registered new tserver with Master: d046a0f9cc58427986a4fa16815e54e6 (127.13.75.130:35027)
I20260923 15:01:37.205710 14730 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.75.130@KRBTEST.COM'} at 127.13.75.130:38195
WARNING: no policy specified for kudu/127.13.75.131@KRBTEST.COM; defaulting to no policy
I20260923 15:01:37.212098 14781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:37.205621 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.130:36357 (local address 127.13.75.190:35253)
0923 15:01:37.205659 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0923 15:01:37.205661 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:01:37.205768 (+ 107us) server_negotiation.cc:407] Connection header received
0923 15:01:37.205866 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:01:37.205867 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:01:37.205877 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:01:37.205906 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:01:37.206379 (+ 473us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:37.207147 (+ 768us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:37.208273 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:37.208527 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:37.208591 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:01:37.209567 (+ 976us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:01:37.209569 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:01:37.209569 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:01:37.209575 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:01:37.210934 (+ 1359us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:01:37.211294 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:01:37.211296 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:01:37.211296 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:01:37.211351 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:01:37.211555 (+ 204us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:01:37.211556 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:01:37.211556 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:01:37.211700 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:01:37.211772 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0923 15:01:37.211930 (+ 158us) server_negotiation.cc:299] Negotiation successful
0923 15:01:37.212006 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260923 15:01:37.212113 15080 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:37.205570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:35253 (local address 127.13.75.130:36357)
0923 15:01:37.205706 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0923 15:01:37.205716 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 15:01:37.205795 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:01:37.205970 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:01:37.205971 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:01:37.206217 (+ 246us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:01:37.206290 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:01:37.206292 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:01:37.207258 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:01:37.207258 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:01:37.208191 (+ 933us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:01:37.208194 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:01:37.208612 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:01:37.208613 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:01:37.208693 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:01:37.209150 (+ 457us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:01:37.209160 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:01:37.209432 (+ 272us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:01:37.211023 (+ 1591us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:01:37.211024 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:01:37.211024 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:01:37.211210 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:01:37.211415 (+ 205us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:01:37.211415 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:01:37.211416 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:01:37.211475 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:01:37.211776 (+ 301us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:01:37.211777 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:01:37.211824 (+ 47us) client_negotiation.cc:769] Sending connection context
0923 15:01:37.211905 (+ 81us) client_negotiation.cc:240] Negotiation successful
0923 15:01:37.212007 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":110,"thread_start_us":69,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.75.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.75.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.131.keytab.
Entry for principal kudu/127.13.75.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.131.keytab.
I20260923 15:01:37.229699 13614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKC4soy/build/release/bin/kudu
/tmp/dist-test-taskKC4soy/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175691491353-13614-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175691491353-13614-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175691491353-13614-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175691491353-13614-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.75.131:0
--local_ip_for_outbound_sockets=127.13.75.131
--webserver_interface=127.13.75.131
--webserver_port=0
--tserver_master_addrs=127.13.75.190:35253
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.131.keytab
--principal=kudu/127.13.75.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.75.148:36005
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175691491353-13614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175691491353-13614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175691491353-13614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:01:37.312778 15082 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:01:37.312983 15082 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:01:37.313006 15082 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:01:37.314175 15082 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:01:37.314220 15082 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:01:37.314242 15082 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260923 15:01:37.314296 15082 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.75.131
I20260923 15:01:37.315827 15082 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.75.148:36005
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175691491353-13614-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175691491353-13614-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.131.keytab
--principal=kudu/127.13.75.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.75.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175691491353-13614-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.75.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.13.75.190:35253
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15082
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.75.131
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175691491353-13614-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:01:37.316073 15082 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:01:37.316298 15082 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:01:37.321148 15088 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:01:37.321200 15090 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:01:37.321156 15087 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:01:37.321322 15082 server_base.cc:1034] running on GCE node
I20260923 15:01:37.321622 15082 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:01:37.321879 15082 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:01:37.323035 15082 hybrid_clock.cc:648] HybridClock initialized: now 1790175697323010 us; error 46 us; skew 500 ppm
Sep 23 15:01:37 dist-test-slave-cnb0 krb5kdc[14685](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175697, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:01:37.325573 15082 init.cc:377] Logged in from keytab as kudu/127.13.75.131@KRBTEST.COM (short username kudu)
I20260923 15:01:37.326041 15082 webserver.cc:460] Webserver started at http://127.13.75.131:36137/ using document root <none> and password file <none>
I20260923 15:01:37.326290 15082 fs_manager.cc:359] Metadata directory not provided
I20260923 15:01:37.326337 15082 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:01:37.326498 15082 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:01:37.327723 15082 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175691491353-13614-0/minicluster-data/ts-2/data/instance:
uuid: "280fe8782f3a420a917580684f455c7a"
format_stamp: "Formatted at 2026-09-23 15:01:37 on dist-test-slave-cnb0"
server_key: "462bbdfad20f81bfb143242471cee7dc"
server_key_iv: "f279e28b68e4c33b0b3f522a1fead3d8"
server_key_version: "encryptionkey@0"
I20260923 15:01:37.328086 15082 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175691491353-13614-0/minicluster-data/ts-2/wal/instance:
uuid: "280fe8782f3a420a917580684f455c7a"
format_stamp: "Formatted at 2026-09-23 15:01:37 on dist-test-slave-cnb0"
server_key: "462bbdfad20f81bfb143242471cee7dc"
server_key_iv: "f279e28b68e4c33b0b3f522a1fead3d8"
server_key_version: "encryptionkey@0"
I20260923 15:01:37.329399 15082 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 15:01:37.330222 15097 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:01:37.330456 15082 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:01:37.330526 15082 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175691491353-13614-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175691491353-13614-0/minicluster-data/ts-2/wal
uuid: "280fe8782f3a420a917580684f455c7a"
format_stamp: "Formatted at 2026-09-23 15:01:37 on dist-test-slave-cnb0"
server_key: "462bbdfad20f81bfb143242471cee7dc"
server_key_iv: "f279e28b68e4c33b0b3f522a1fead3d8"
server_key_version: "encryptionkey@0"
I20260923 15:01:37.330613 15082 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175691491353-13614-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175691491353-13614-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175691491353-13614-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:01:37.339491 15082 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:01:37.341578 15082 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:01:37.341729 15082 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:01:37.342240 15082 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 15:01:37.342303 15082 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 15:01:37.342347 15082 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 15:01:37.342378 15082 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 15:01:37.349303 15082 rpc_server.cc:307] RPC server started. Bound to: 127.13.75.131:34765
I20260923 15:01:37.349427 15212 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.75.131:34765 every 8 connection(s)
Sep 23 15:01:37 dist-test-slave-cnb0 krb5kdc[14685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175697, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.131@KRBTEST.COM for kudu/127.13.75.190@KRBTEST.COM
I20260923 15:01:37.349759 15082 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175691491353-13614-0/minicluster-data/ts-2/data/info.pb
I20260923 15:01:37.353036 14781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:37.342787 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.131:43949 (local address 127.13.75.190:35253)
0923 15:01:37.342827 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0923 15:01:37.342830 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:01:37.343093 (+ 263us) server_negotiation.cc:407] Connection header received
0923 15:01:37.343629 (+ 536us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:01:37.343630 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:01:37.343644 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:01:37.343699 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:01:37.344553 (+ 854us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:37.345411 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:37.346489 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:37.346808 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:37.346901 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:01:37.350133 (+ 3232us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:01:37.350134 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:01:37.350135 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:01:37.350143 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:01:37.351644 (+ 1501us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:01:37.352070 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:01:37.352071 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:01:37.352071 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:01:37.352134 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:01:37.352382 (+ 248us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:01:37.352382 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:01:37.352383 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:01:37.352512 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:01:37.352584 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0923 15:01:37.352859 (+ 275us) server_negotiation.cc:299] Negotiation successful
0923 15:01:37.352945 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260923 15:01:37.353227 15106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:37.342795 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:35253 (local address 127.13.75.131:43949)
0923 15:01:37.342979 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0923 15:01:37.343014 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0923 15:01:37.343528 (+ 514us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:01:37.343773 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:01:37.343777 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:01:37.344161 (+ 384us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:01:37.344446 (+ 285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:01:37.344453 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:01:37.345526 (+ 1073us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:01:37.345527 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:01:37.346400 (+ 873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:01:37.346402 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:01:37.346941 (+ 539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:01:37.346943 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:01:37.347119 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:01:37.348231 (+ 1112us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:01:37.348243 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:01:37.350042 (+ 1799us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:01:37.351744 (+ 1702us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:01:37.351751 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:01:37.351753 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:01:37.351982 (+ 229us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:01:37.352199 (+ 217us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:01:37.352199 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:01:37.352200 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:01:37.352312 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:01:37.352575 (+ 263us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:01:37.352581 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:01:37.352721 (+ 140us) client_negotiation.cc:769] Sending connection context
0923 15:01:37.352821 (+ 100us) client_negotiation.cc:240] Negotiation successful
0923 15:01:37.352994 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":110,"threads_started":1}
I20260923 15:01:37.353703 15213 heartbeater.cc:347] Connected to a master server at 127.13.75.190:35253
I20260923 15:01:37.353783 15213 heartbeater.cc:464] Registering TS with master...
I20260923 15:01:37.353977 15213 heartbeater.cc:511] Master 127.13.75.190:35253 requested a full tablet report, sending...
I20260923 15:01:37.354384 13614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKC4soy/build/release/bin/kudu as pid 15082
I20260923 15:01:37.354461 13614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175691491353-13614-0/minicluster-data/ts-2/wal/instance
I20260923 15:01:37.354655 13614 external_mini_cluster.cc:1428] Setting key 6c0197d0f825ab959b690e0e5be4cdf6
I20260923 15:01:37.354949 14729 ts_manager.cc:194] Registered new tserver with Master: 280fe8782f3a420a917580684f455c7a (127.13.75.131:34765)
I20260923 15:01:37.355435 13614 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260923 15:01:37.355492 14729 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.75.131@KRBTEST.COM'} at 127.13.75.131:43949
I20260923 15:01:37.360862 15219 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:37.354827 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:35253 (local address 127.13.75.131:46615)
0923 15:01:37.355063 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0923 15:01:37.355074 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 15:01:37.355148 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:01:37.355269 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:01:37.355269 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:01:37.355477 (+ 208us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:01:37.355546 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:01:37.355548 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:01:37.356433 (+ 885us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:01:37.356434 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:01:37.356932 (+ 498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:01:37.356933 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:01:37.357290 (+ 357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:01:37.357291 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:01:37.357366 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:01:37.357790 (+ 424us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:01:37.357794 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:01:37.358129 (+ 335us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:01:37.359733 (+ 1604us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:01:37.359735 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:01:37.359735 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:01:37.359965 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:01:37.360194 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:01:37.360195 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:01:37.360195 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:01:37.360251 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:01:37.360517 (+ 266us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:01:37.360518 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:01:37.360583 (+ 65us) client_negotiation.cc:769] Sending connection context
0923 15:01:37.360663 (+ 80us) client_negotiation.cc:240] Negotiation successful
0923 15:01:37.360767 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":128,"threads_started":1}
I20260923 15:01:37.360857 14781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:37.354833 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.131:46615 (local address 127.13.75.190:35253)
0923 15:01:37.354872 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0923 15:01:37.354874 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:01:37.355123 (+ 249us) server_negotiation.cc:407] Connection header received
0923 15:01:37.355201 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:01:37.355202 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:01:37.355212 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:01:37.355266 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:01:37.355646 (+ 380us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:37.356326 (+ 680us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:37.357017 (+ 691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:37.357220 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:37.357291 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:01:37.358209 (+ 918us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:01:37.358210 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:01:37.358211 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:01:37.358216 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:01:37.359658 (+ 1442us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:01:37.360043 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:01:37.360044 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:01:37.360044 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:01:37.360123 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:01:37.360322 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:01:37.360323 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:01:37.360323 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:01:37.360442 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:01:37.360528 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0923 15:01:37.360689 (+ 161us) server_negotiation.cc:299] Negotiation successful
0923 15:01:37.360765 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
Sep 23 15:01:37 dist-test-slave-cnb0 krb5kdc[14685](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175697, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 23 15:01:37 dist-test-slave-cnb0 krb5kdc[14685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175697, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.13.75.190@KRBTEST.COM
I20260923 15:01:37.379593 14781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:37.372441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45296 (local address 127.13.75.190:35253)
0923 15:01:37.372507 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0923 15:01:37.372509 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:01:37.372518 (+ 9us) server_negotiation.cc:407] Connection header received
0923 15:01:37.372582 (+ 64us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:01:37.372583 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:01:37.372593 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:01:37.372647 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:01:37.373331 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:37.373831 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:37.374583 (+ 752us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:37.374767 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:37.374819 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:01:37.377086 (+ 2267us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:01:37.377088 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:01:37.377088 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:01:37.377096 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:01:37.378483 (+ 1387us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:01:37.378918 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:01:37.378919 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:01:37.378919 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:01:37.378984 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:01:37.379152 (+ 168us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:01:37.379152 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:01:37.379153 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:01:37.379259 (+ 106us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:01:37.379334 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0923 15:01:37.379419 (+ 85us) server_negotiation.cc:299] Negotiation successful
0923 15:01:37.379480 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260923 15:01:37.381323 14729 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45296:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 15:01:37.382975 14729 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 15:01:37.390020 15235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:37.386039 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49888 (local address 127.13.75.130:35027)
0923 15:01:37.386196 (+ 157us) server_negotiation.cc:206] Beginning negotiation
0923 15:01:37.386201 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 15:01:37.386211 (+ 10us) server_negotiation.cc:407] Connection header received
0923 15:01:37.386252 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:01:37.386254 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:01:37.386269 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:01:37.386343 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:01:37.386662 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:37.387796 (+ 1134us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:37.389038 (+ 1242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:37.389538 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:37.389629 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:01:37.389691 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0923 15:01:37.389853 (+ 162us) server_negotiation.cc:299] Negotiation successful
0923 15:01:37.389895 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":84,"threads_started":1}
I20260923 15:01:37.390009 15232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:37.385811 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.130:35027 (local address 127.0.0.1:49888)
0923 15:01:37.386068 (+ 257us) negotiation.cc:107] Waiting for socket to connect
0923 15:01:37.386078 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 15:01:37.386119 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:01:37.386407 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:01:37.386408 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:01:37.386409 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:01:37.386587 (+ 178us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:01:37.386588 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:01:37.387896 (+ 1308us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:01:37.387897 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:01:37.388956 (+ 1059us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:01:37.388958 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:01:37.389637 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:01:37.389638 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:01:37.389726 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:01:37.389745 (+ 19us) client_negotiation.cc:769] Sending connection context
0923 15:01:37.389833 (+ 88us) client_negotiation.cc:240] Negotiation successful
0923 15:01:37.389894 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":238,"mutex_wait_us":31,"thread_start_us":68,"threads_started":1}
I20260923 15:01:37.390194 15234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:37.385807 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52236 (local address 127.13.75.129:43089)
0923 15:01:37.386004 (+ 197us) server_negotiation.cc:206] Beginning negotiation
0923 15:01:37.386010 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 15:01:37.386022 (+ 12us) server_negotiation.cc:407] Connection header received
0923 15:01:37.386178 (+ 156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:01:37.386182 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:01:37.386197 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:01:37.386307 (+ 110us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:01:37.386569 (+ 262us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:37.387382 (+ 813us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:37.388774 (+ 1392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:37.389557 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:37.389629 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:01:37.389690 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0923 15:01:37.390057 (+ 367us) server_negotiation.cc:299] Negotiation successful
0923 15:01:37.390099 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":165,"thread_start_us":90,"threads_started":1}
I20260923 15:01:37.390278 15231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:37.385711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51358 (local address 127.13.75.131:34765)
0923 15:01:37.386229 (+ 518us) server_negotiation.cc:206] Beginning negotiation
0923 15:01:37.386232 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:01:37.386241 (+ 9us) server_negotiation.cc:407] Connection header received
0923 15:01:37.386276 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:01:37.386278 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:01:37.386294 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:01:37.386471 (+ 177us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:01:37.386574 (+ 103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:37.387821 (+ 1247us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:37.388968 (+ 1147us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:37.389670 (+ 702us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:37.390046 (+ 376us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:01:37.390107 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0923 15:01:37.390131 (+ 24us) server_negotiation.cc:299] Negotiation successful
0923 15:01:37.390174 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":487,"thread_start_us":96,"threads_started":1}
I20260923 15:01:37.390019 15233 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:37.385703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.129:43089 (local address 127.0.0.1:52236)
0923 15:01:37.385922 (+ 219us) negotiation.cc:107] Waiting for socket to connect
0923 15:01:37.385959 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0923 15:01:37.386031 (+ 72us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:01:37.386388 (+ 357us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:01:37.386391 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:01:37.386394 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:01:37.386486 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:01:37.386489 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:01:37.387482 (+ 993us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:01:37.387484 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:01:37.388688 (+ 1204us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:01:37.388691 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:01:37.389727 (+ 1036us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:01:37.389729 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:01:37.389803 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:01:37.389807 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 15:01:37.389856 (+ 49us) client_negotiation.cc:240] Negotiation successful
0923 15:01:37.389904 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":197,"mutex_wait_us":4,"thread_start_us":111,"threads_started":1}
I20260923 15:01:37.390019 15230 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:37.385597 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.131:34765 (local address 127.0.0.1:51358)
0923 15:01:37.385914 (+ 317us) negotiation.cc:107] Waiting for socket to connect
0923 15:01:37.385960 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0923 15:01:37.386031 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:01:37.386401 (+ 370us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:01:37.386403 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:01:37.386405 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:01:37.386500 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:01:37.386502 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:01:37.387925 (+ 1423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:01:37.387926 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:01:37.388875 (+ 949us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:01:37.388878 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:01:37.389753 (+ 875us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:01:37.389754 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:01:37.389805 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:01:37.389808 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 15:01:37.389839 (+ 31us) client_negotiation.cc:240] Negotiation successful
0923 15:01:37.389903 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":287,"mutex_wait_us":30,"thread_start_us":72,"threads_started":1}
I20260923 15:01:37.391207 15007 tablet_service.cc:1467] Processing CreateTablet for tablet 7a66ceff1a2e452b96013c3b10c77faa (DEFAULT_TABLE table=test-table [id=5d365a222c2d44e8ace335d760ce93f7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 15:01:37.391212 14867 tablet_service.cc:1467] Processing CreateTablet for tablet 7a66ceff1a2e452b96013c3b10c77faa (DEFAULT_TABLE table=test-table [id=5d365a222c2d44e8ace335d760ce93f7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 15:01:37.391526 14867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7a66ceff1a2e452b96013c3b10c77faa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:01:37.391551 15007 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7a66ceff1a2e452b96013c3b10c77faa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:01:37.394421 15236 tablet_bootstrap.cc:492] T 7a66ceff1a2e452b96013c3b10c77faa P 1cebda21ec684fcabfa5beaef4edda08: Bootstrap starting.
I20260923 15:01:37.394420 15237 tablet_bootstrap.cc:492] T 7a66ceff1a2e452b96013c3b10c77faa P d046a0f9cc58427986a4fa16815e54e6: Bootstrap starting.
I20260923 15:01:37.395077 15147 tablet_service.cc:1467] Processing CreateTablet for tablet 7a66ceff1a2e452b96013c3b10c77faa (DEFAULT_TABLE table=test-table [id=5d365a222c2d44e8ace335d760ce93f7]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 15:01:37.395210 15236 tablet_bootstrap.cc:654] T 7a66ceff1a2e452b96013c3b10c77faa P 1cebda21ec684fcabfa5beaef4edda08: Neither blocks nor log segments found. Creating new log.
I20260923 15:01:37.395236 15237 tablet_bootstrap.cc:654] T 7a66ceff1a2e452b96013c3b10c77faa P d046a0f9cc58427986a4fa16815e54e6: Neither blocks nor log segments found. Creating new log.
I20260923 15:01:37.395421 15147 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 7a66ceff1a2e452b96013c3b10c77faa. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:01:37.395582 15237 log.cc:824] T 7a66ceff1a2e452b96013c3b10c77faa P d046a0f9cc58427986a4fa16815e54e6: Log is configured to *not* fsync() on all Append() calls
I20260923 15:01:37.395670 15236 log.cc:824] T 7a66ceff1a2e452b96013c3b10c77faa P 1cebda21ec684fcabfa5beaef4edda08: Log is configured to *not* fsync() on all Append() calls
I20260923 15:01:37.396410 15236 tablet_bootstrap.cc:492] T 7a66ceff1a2e452b96013c3b10c77faa P 1cebda21ec684fcabfa5beaef4edda08: No bootstrap required, opened a new log
I20260923 15:01:37.396486 15236 ts_tablet_manager.cc:1397] T 7a66ceff1a2e452b96013c3b10c77faa P 1cebda21ec684fcabfa5beaef4edda08: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 15:01:37.398689 15240 tablet_bootstrap.cc:492] T 7a66ceff1a2e452b96013c3b10c77faa P 280fe8782f3a420a917580684f455c7a: Bootstrap starting.
I20260923 15:01:37.398743 15236 raft_consensus.cc:348] T 7a66ceff1a2e452b96013c3b10c77faa P 1cebda21ec684fcabfa5beaef4edda08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "280fe8782f3a420a917580684f455c7a" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 34765 } } peers { permanent_uuid: "d046a0f9cc58427986a4fa16815e54e6" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 35027 } } peers { permanent_uuid: "1cebda21ec684fcabfa5beaef4edda08" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 43089 } }
I20260923 15:01:37.398965 15236 raft_consensus.cc:374] T 7a66ceff1a2e452b96013c3b10c77faa P 1cebda21ec684fcabfa5beaef4edda08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:01:37.399008 15236 raft_consensus.cc:729] T 7a66ceff1a2e452b96013c3b10c77faa P 1cebda21ec684fcabfa5beaef4edda08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1cebda21ec684fcabfa5beaef4edda08, State: Initialized, Role: FOLLOWER
I20260923 15:01:37.399140 15236 consensus_queue.cc:260] T 7a66ceff1a2e452b96013c3b10c77faa P 1cebda21ec684fcabfa5beaef4edda08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "280fe8782f3a420a917580684f455c7a" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 34765 } } peers { permanent_uuid: "d046a0f9cc58427986a4fa16815e54e6" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 35027 } } peers { permanent_uuid: "1cebda21ec684fcabfa5beaef4edda08" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 43089 } }
I20260923 15:01:37.399210 15237 tablet_bootstrap.cc:492] T 7a66ceff1a2e452b96013c3b10c77faa P d046a0f9cc58427986a4fa16815e54e6: No bootstrap required, opened a new log
I20260923 15:01:37.399271 15237 ts_tablet_manager.cc:1397] T 7a66ceff1a2e452b96013c3b10c77faa P d046a0f9cc58427986a4fa16815e54e6: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260923 15:01:37.399430 15236 ts_tablet_manager.cc:1428] T 7a66ceff1a2e452b96013c3b10c77faa P 1cebda21ec684fcabfa5beaef4edda08: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260923 15:01:37.399557 14933 heartbeater.cc:503] Master 127.13.75.190:35253 was elected leader, sending a full tablet report...
I20260923 15:01:37.399771 15240 tablet_bootstrap.cc:654] T 7a66ceff1a2e452b96013c3b10c77faa P 280fe8782f3a420a917580684f455c7a: Neither blocks nor log segments found. Creating new log.
I20260923 15:01:37.400134 15240 log.cc:824] T 7a66ceff1a2e452b96013c3b10c77faa P 280fe8782f3a420a917580684f455c7a: Log is configured to *not* fsync() on all Append() calls
I20260923 15:01:37.400893 15240 tablet_bootstrap.cc:492] T 7a66ceff1a2e452b96013c3b10c77faa P 280fe8782f3a420a917580684f455c7a: No bootstrap required, opened a new log
I20260923 15:01:37.401011 15240 ts_tablet_manager.cc:1397] T 7a66ceff1a2e452b96013c3b10c77faa P 280fe8782f3a420a917580684f455c7a: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260923 15:01:37.401294 15237 raft_consensus.cc:348] T 7a66ceff1a2e452b96013c3b10c77faa P d046a0f9cc58427986a4fa16815e54e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "280fe8782f3a420a917580684f455c7a" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 34765 } } peers { permanent_uuid: "d046a0f9cc58427986a4fa16815e54e6" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 35027 } } peers { permanent_uuid: "1cebda21ec684fcabfa5beaef4edda08" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 43089 } }
I20260923 15:01:37.401434 15237 raft_consensus.cc:374] T 7a66ceff1a2e452b96013c3b10c77faa P d046a0f9cc58427986a4fa16815e54e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:01:37.401482 15237 raft_consensus.cc:729] T 7a66ceff1a2e452b96013c3b10c77faa P d046a0f9cc58427986a4fa16815e54e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d046a0f9cc58427986a4fa16815e54e6, State: Initialized, Role: FOLLOWER
I20260923 15:01:37.401599 15237 consensus_queue.cc:260] T 7a66ceff1a2e452b96013c3b10c77faa P d046a0f9cc58427986a4fa16815e54e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "280fe8782f3a420a917580684f455c7a" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 34765 } } peers { permanent_uuid: "d046a0f9cc58427986a4fa16815e54e6" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 35027 } } peers { permanent_uuid: "1cebda21ec684fcabfa5beaef4edda08" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 43089 } }
I20260923 15:01:37.401870 15237 ts_tablet_manager.cc:1428] T 7a66ceff1a2e452b96013c3b10c77faa P d046a0f9cc58427986a4fa16815e54e6: Time spent starting tablet: real 0.003s user 0.001s sys 0.002s
I20260923 15:01:37.402020 15073 heartbeater.cc:503] Master 127.13.75.190:35253 was elected leader, sending a full tablet report...
I20260923 15:01:37.403249 15240 raft_consensus.cc:348] T 7a66ceff1a2e452b96013c3b10c77faa P 280fe8782f3a420a917580684f455c7a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "280fe8782f3a420a917580684f455c7a" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 34765 } } peers { permanent_uuid: "d046a0f9cc58427986a4fa16815e54e6" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 35027 } } peers { permanent_uuid: "1cebda21ec684fcabfa5beaef4edda08" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 43089 } }
I20260923 15:01:37.403378 15240 raft_consensus.cc:374] T 7a66ceff1a2e452b96013c3b10c77faa P 280fe8782f3a420a917580684f455c7a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:01:37.403410 15240 raft_consensus.cc:729] T 7a66ceff1a2e452b96013c3b10c77faa P 280fe8782f3a420a917580684f455c7a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 280fe8782f3a420a917580684f455c7a, State: Initialized, Role: FOLLOWER
I20260923 15:01:37.403534 15240 consensus_queue.cc:260] T 7a66ceff1a2e452b96013c3b10c77faa P 280fe8782f3a420a917580684f455c7a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "280fe8782f3a420a917580684f455c7a" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 34765 } } peers { permanent_uuid: "d046a0f9cc58427986a4fa16815e54e6" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 35027 } } peers { permanent_uuid: "1cebda21ec684fcabfa5beaef4edda08" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 43089 } }
I20260923 15:01:37.403775 15240 ts_tablet_manager.cc:1428] T 7a66ceff1a2e452b96013c3b10c77faa P 280fe8782f3a420a917580684f455c7a: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260923 15:01:37.403846 15213 heartbeater.cc:503] Master 127.13.75.190:35253 was elected leader, sending a full tablet report...
W20260923 15:01:37.451349 15074 tablet.cc:2369] T 7a66ceff1a2e452b96013c3b10c77faa P d046a0f9cc58427986a4fa16815e54e6: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 15:01:37.491466 15244 raft_consensus.cc:482] T 7a66ceff1a2e452b96013c3b10c77faa P 280fe8782f3a420a917580684f455c7a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 15:01:37.491601 15244 raft_consensus.cc:504] T 7a66ceff1a2e452b96013c3b10c77faa P 280fe8782f3a420a917580684f455c7a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "280fe8782f3a420a917580684f455c7a" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 34765 } } peers { permanent_uuid: "d046a0f9cc58427986a4fa16815e54e6" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 35027 } } peers { permanent_uuid: "1cebda21ec684fcabfa5beaef4edda08" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 43089 } }
I20260923 15:01:37.492025 15244 leader_election.cc:288] T 7a66ceff1a2e452b96013c3b10c77faa P 280fe8782f3a420a917580684f455c7a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d046a0f9cc58427986a4fa16815e54e6 (127.13.75.130:35027), 1cebda21ec684fcabfa5beaef4edda08 (127.13.75.129:43089)
I20260923 15:01:37.496060 15234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:37.492422 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.131:54083 (local address 127.13.75.129:43089)
0923 15:01:37.492491 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0923 15:01:37.492493 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:01:37.492504 (+ 11us) server_negotiation.cc:407] Connection header received
0923 15:01:37.492538 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:01:37.492538 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:01:37.492549 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:01:37.492621 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:01:37.492821 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:37.493628 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:37.495104 (+ 1476us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:37.495666 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:37.495750 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:01:37.495760 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0923 15:01:37.495939 (+ 179us) server_negotiation.cc:299] Negotiation successful
0923 15:01:37.495982 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260923 15:01:37.496060 15235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:37.492348 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.131:60627 (local address 127.13.75.130:35027)
0923 15:01:37.492402 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0923 15:01:37.492404 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:01:37.492793 (+ 389us) server_negotiation.cc:407] Connection header received
0923 15:01:37.492826 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:01:37.492827 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:01:37.492838 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:01:37.492908 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:01:37.493041 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:37.493714 (+ 673us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:37.495104 (+ 1390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:37.495683 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:37.495752 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:01:37.495763 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0923 15:01:37.495939 (+ 176us) server_negotiation.cc:299] Negotiation successful
0923 15:01:37.495982 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260923 15:01:37.496074 15106 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:37.492264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.129:43089 (local address 127.13.75.131:54083)
0923 15:01:37.492331 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0923 15:01:37.492345 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0923 15:01:37.492415 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:01:37.492639 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:01:37.492640 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:01:37.492642 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:01:37.492723 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:01:37.492725 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:01:37.493704 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:01:37.493705 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:01:37.494998 (+ 1293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:01:37.495001 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:01:37.495758 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:01:37.495759 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:01:37.495861 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:01:37.495864 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 15:01:37.495921 (+ 57us) client_negotiation.cc:240] Negotiation successful
0923 15:01:37.495977 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42,"mutex_wait_us":23}
I20260923 15:01:37.496083 15245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:37.492264 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.130:35027 (local address 127.13.75.131:60627)
0923 15:01:37.492658 (+ 394us) negotiation.cc:107] Waiting for socket to connect
0923 15:01:37.492668 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 15:01:37.492728 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:01:37.492916 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:01:37.492916 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:01:37.492918 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:01:37.492973 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:01:37.492974 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:01:37.493782 (+ 808us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:01:37.493783 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:01:37.494998 (+ 1215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:01:37.495001 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:01:37.495758 (+ 757us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:01:37.495759 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:01:37.495856 (+ 97us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:01:37.495860 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 15:01:37.495921 (+ 61us) client_negotiation.cc:240] Negotiation successful
0923 15:01:37.495977 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":375,"mutex_wait_us":170,"thread_start_us":142,"threads_started":1}
I20260923 15:01:37.496431 14887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7a66ceff1a2e452b96013c3b10c77faa" candidate_uuid: "280fe8782f3a420a917580684f455c7a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1cebda21ec684fcabfa5beaef4edda08" is_pre_election: true
I20260923 15:01:37.496467 15027 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7a66ceff1a2e452b96013c3b10c77faa" candidate_uuid: "280fe8782f3a420a917580684f455c7a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d046a0f9cc58427986a4fa16815e54e6" is_pre_election: true
I20260923 15:01:37.496610 14887 raft_consensus.cc:2454] T 7a66ceff1a2e452b96013c3b10c77faa P 1cebda21ec684fcabfa5beaef4edda08 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 280fe8782f3a420a917580684f455c7a in term 0.
I20260923 15:01:37.496629 15027 raft_consensus.cc:2454] T 7a66ceff1a2e452b96013c3b10c77faa P d046a0f9cc58427986a4fa16815e54e6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 280fe8782f3a420a917580684f455c7a in term 0.
I20260923 15:01:37.496841 15100 leader_election.cc:302] T 7a66ceff1a2e452b96013c3b10c77faa P 280fe8782f3a420a917580684f455c7a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1cebda21ec684fcabfa5beaef4edda08, 280fe8782f3a420a917580684f455c7a; no voters:
I20260923 15:01:37.496992 15244 raft_consensus.cc:2781] T 7a66ceff1a2e452b96013c3b10c77faa P 280fe8782f3a420a917580684f455c7a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 15:01:37.497057 15244 raft_consensus.cc:482] T 7a66ceff1a2e452b96013c3b10c77faa P 280fe8782f3a420a917580684f455c7a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 15:01:37.497076 15244 raft_consensus.cc:3037] T 7a66ceff1a2e452b96013c3b10c77faa P 280fe8782f3a420a917580684f455c7a [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:01:37.497745 15244 raft_consensus.cc:504] T 7a66ceff1a2e452b96013c3b10c77faa P 280fe8782f3a420a917580684f455c7a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "280fe8782f3a420a917580684f455c7a" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 34765 } } peers { permanent_uuid: "d046a0f9cc58427986a4fa16815e54e6" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 35027 } } peers { permanent_uuid: "1cebda21ec684fcabfa5beaef4edda08" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 43089 } }
I20260923 15:01:37.497881 15244 leader_election.cc:288] T 7a66ceff1a2e452b96013c3b10c77faa P 280fe8782f3a420a917580684f455c7a [CANDIDATE]: Term 1 election: Requested vote from peers d046a0f9cc58427986a4fa16815e54e6 (127.13.75.130:35027), 1cebda21ec684fcabfa5beaef4edda08 (127.13.75.129:43089)
I20260923 15:01:37.498023 15027 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7a66ceff1a2e452b96013c3b10c77faa" candidate_uuid: "280fe8782f3a420a917580684f455c7a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d046a0f9cc58427986a4fa16815e54e6"
I20260923 15:01:37.498050 14887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "7a66ceff1a2e452b96013c3b10c77faa" candidate_uuid: "280fe8782f3a420a917580684f455c7a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1cebda21ec684fcabfa5beaef4edda08"
I20260923 15:01:37.498112 14887 raft_consensus.cc:3037] T 7a66ceff1a2e452b96013c3b10c77faa P 1cebda21ec684fcabfa5beaef4edda08 [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:01:37.498108 15027 raft_consensus.cc:3037] T 7a66ceff1a2e452b96013c3b10c77faa P d046a0f9cc58427986a4fa16815e54e6 [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:01:37.499060 14887 raft_consensus.cc:2454] T 7a66ceff1a2e452b96013c3b10c77faa P 1cebda21ec684fcabfa5beaef4edda08 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 280fe8782f3a420a917580684f455c7a in term 1.
I20260923 15:01:37.499058 15027 raft_consensus.cc:2454] T 7a66ceff1a2e452b96013c3b10c77faa P d046a0f9cc58427986a4fa16815e54e6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 280fe8782f3a420a917580684f455c7a in term 1.
I20260923 15:01:37.499220 15100 leader_election.cc:302] T 7a66ceff1a2e452b96013c3b10c77faa P 280fe8782f3a420a917580684f455c7a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1cebda21ec684fcabfa5beaef4edda08, 280fe8782f3a420a917580684f455c7a; no voters:
I20260923 15:01:37.499306 15244 raft_consensus.cc:2781] T 7a66ceff1a2e452b96013c3b10c77faa P 280fe8782f3a420a917580684f455c7a [term 1 FOLLOWER]: Leader election won for term 1
I20260923 15:01:37.499468 15244 raft_consensus.cc:686] T 7a66ceff1a2e452b96013c3b10c77faa P 280fe8782f3a420a917580684f455c7a [term 1 LEADER]: Becoming Leader. State: Replica: 280fe8782f3a420a917580684f455c7a, State: Running, Role: LEADER
I20260923 15:01:37.499577 15244 consensus_queue.cc:237] T 7a66ceff1a2e452b96013c3b10c77faa P 280fe8782f3a420a917580684f455c7a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "280fe8782f3a420a917580684f455c7a" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 34765 } } peers { permanent_uuid: "d046a0f9cc58427986a4fa16815e54e6" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 35027 } } peers { permanent_uuid: "1cebda21ec684fcabfa5beaef4edda08" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 43089 } }
I20260923 15:01:37.500612 14729 catalog_manager.cc:5495] T 7a66ceff1a2e452b96013c3b10c77faa P 280fe8782f3a420a917580684f455c7a reported cstate change: term changed from 0 to 1, leader changed from <none> to 280fe8782f3a420a917580684f455c7a (127.13.75.131). New cstate: current_term: 1 leader_uuid: "280fe8782f3a420a917580684f455c7a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "280fe8782f3a420a917580684f455c7a" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 34765 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d046a0f9cc58427986a4fa16815e54e6" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 35027 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "1cebda21ec684fcabfa5beaef4edda08" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 43089 } health_report { overall_health: UNKNOWN } } }
I20260923 15:01:37.536438 15231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:37.533781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51362 (local address 127.13.75.131:34765)
0923 15:01:37.533831 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0923 15:01:37.533834 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:01:37.533844 (+ 10us) server_negotiation.cc:407] Connection header received
0923 15:01:37.533971 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:01:37.533972 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:01:37.533989 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:01:37.534060 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 15:01:37.534218 (+ 158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:37.534703 (+ 485us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:37.535575 (+ 872us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:37.535770 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:37.535862 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:01:37.536074 (+ 212us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 15:01:37.536171 (+ 97us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 15:01:37.536255 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0923 15:01:37.536299 (+ 44us) server_negotiation.cc:299] Negotiation successful
0923 15:01:37.536322 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260923 15:01:37.538261 14887 raft_consensus.cc:1261] T 7a66ceff1a2e452b96013c3b10c77faa P 1cebda21ec684fcabfa5beaef4edda08 [term 1 FOLLOWER]: Refusing update from remote peer 280fe8782f3a420a917580684f455c7a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 15:01:37.538260 15027 raft_consensus.cc:1261] T 7a66ceff1a2e452b96013c3b10c77faa P d046a0f9cc58427986a4fa16815e54e6 [term 1 FOLLOWER]: Refusing update from remote peer 280fe8782f3a420a917580684f455c7a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 15:01:37.538619 15244 consensus_queue.cc:1035] T 7a66ceff1a2e452b96013c3b10c77faa P 280fe8782f3a420a917580684f455c7a [LEADER]: Connected to new peer: Peer: permanent_uuid: "d046a0f9cc58427986a4fa16815e54e6" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 35027 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 15:01:37.538792 15246 consensus_queue.cc:1035] T 7a66ceff1a2e452b96013c3b10c77faa P 280fe8782f3a420a917580684f455c7a [LEADER]: Connected to new peer: Peer: permanent_uuid: "1cebda21ec684fcabfa5beaef4edda08" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 43089 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 15:01:37.541292 15249 mvcc.cc:205] Tried to move back new op lower bound from 7332559657113964544 to 7332559656959066112. Current Snapshot: MvccSnapshot[applied={T|T < 7332559657113964544}]
I20260923 15:01:37.541458 15250 mvcc.cc:205] Tried to move back new op lower bound from 7332559657113964544 to 7332559656959066112. Current Snapshot: MvccSnapshot[applied={T|T < 7332559657113964544}]
I20260923 15:01:37.545387 14728 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45296:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 15:01:37.545540 14728 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45296:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 15:01:37.546366 14728 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 61f4472f8f444f179d09ce5ce961925c: Sending DeleteTablet for 3 replicas of tablet 7a66ceff1a2e452b96013c3b10c77faa
I20260923 15:01:37.546718 15147 tablet_service.cc:1514] Processing DeleteTablet for tablet 7a66ceff1a2e452b96013c3b10c77faa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 15:01:37 UTC) from {username='kudu', principal='kudu/127.13.75.190@KRBTEST.COM'} at 127.0.0.1:51358
I20260923 15:01:37.546752 15007 tablet_service.cc:1514] Processing DeleteTablet for tablet 7a66ceff1a2e452b96013c3b10c77faa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 15:01:37 UTC) from {username='kudu', principal='kudu/127.13.75.190@KRBTEST.COM'} at 127.0.0.1:49888
I20260923 15:01:37.546954 14867 tablet_service.cc:1514] Processing DeleteTablet for tablet 7a66ceff1a2e452b96013c3b10c77faa with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 15:01:37 UTC) from {username='kudu', principal='kudu/127.13.75.190@KRBTEST.COM'} at 127.0.0.1:52236
I20260923 15:01:37.547098 14728 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45296:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 15:01:37.547264 14728 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 15:01:37.547292 15260 tablet_replica.cc:330] T 7a66ceff1a2e452b96013c3b10c77faa P d046a0f9cc58427986a4fa16815e54e6: stopping tablet replica
I20260923 15:01:37.547395 15260 raft_consensus.cc:2229] T 7a66ceff1a2e452b96013c3b10c77faa P d046a0f9cc58427986a4fa16815e54e6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 15:01:37.547529 15259 tablet_replica.cc:330] T 7a66ceff1a2e452b96013c3b10c77faa P 280fe8782f3a420a917580684f455c7a: stopping tablet replica
I20260923 15:01:37.547540 15260 raft_consensus.cc:2258] T 7a66ceff1a2e452b96013c3b10c77faa P d046a0f9cc58427986a4fa16815e54e6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 15:01:37.547736 15261 tablet_replica.cc:330] T 7a66ceff1a2e452b96013c3b10c77faa P 1cebda21ec684fcabfa5beaef4edda08: stopping tablet replica
I20260923 15:01:37.547763 15259 raft_consensus.cc:2229] T 7a66ceff1a2e452b96013c3b10c77faa P 280fe8782f3a420a917580684f455c7a [term 1 LEADER]: Raft consensus shutting down.
I20260923 15:01:37.547845 15261 raft_consensus.cc:2229] T 7a66ceff1a2e452b96013c3b10c77faa P 1cebda21ec684fcabfa5beaef4edda08 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 15:01:37.547945 15261 raft_consensus.cc:2258] T 7a66ceff1a2e452b96013c3b10c77faa P 1cebda21ec684fcabfa5beaef4edda08 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 15:01:37.548090 15259 raft_consensus.cc:2258] T 7a66ceff1a2e452b96013c3b10c77faa P 280fe8782f3a420a917580684f455c7a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 15:01:37.548367 15260 ts_tablet_manager.cc:1905] T 7a66ceff1a2e452b96013c3b10c77faa P d046a0f9cc58427986a4fa16815e54e6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 15:01:37.548507 14867 tablet_service.cc:1467] Processing CreateTablet for tablet 750be79c62374baea26e714cf95fae21 (DEFAULT_TABLE table=test-table [id=2bd7255e334d4240aa2a25663d5a094f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 15:01:37.548628 14867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 750be79c62374baea26e714cf95fae21. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:01:37.548719 15007 tablet_service.cc:1467] Processing CreateTablet for tablet 750be79c62374baea26e714cf95fae21 (DEFAULT_TABLE table=test-table [id=2bd7255e334d4240aa2a25663d5a094f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 15:01:37.548817 15007 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 750be79c62374baea26e714cf95fae21. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:01:37.549121 15259 ts_tablet_manager.cc:1905] T 7a66ceff1a2e452b96013c3b10c77faa P 280fe8782f3a420a917580684f455c7a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 15:01:37.550140 15237 tablet_bootstrap.cc:492] T 750be79c62374baea26e714cf95fae21 P d046a0f9cc58427986a4fa16815e54e6: Bootstrap starting.
I20260923 15:01:37.550176 15147 tablet_service.cc:1467] Processing CreateTablet for tablet 750be79c62374baea26e714cf95fae21 (DEFAULT_TABLE table=test-table [id=2bd7255e334d4240aa2a25663d5a094f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 15:01:37.550271 15147 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 750be79c62374baea26e714cf95fae21. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:01:37.550563 15237 tablet_bootstrap.cc:654] T 750be79c62374baea26e714cf95fae21 P d046a0f9cc58427986a4fa16815e54e6: Neither blocks nor log segments found. Creating new log.
I20260923 15:01:37.550910 15236 tablet_bootstrap.cc:492] T 750be79c62374baea26e714cf95fae21 P 1cebda21ec684fcabfa5beaef4edda08: Bootstrap starting.
I20260923 15:01:37.550917 15259 ts_tablet_manager.cc:1918] T 7a66ceff1a2e452b96013c3b10c77faa P 280fe8782f3a420a917580684f455c7a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 15:01:37.550976 15259 log.cc:1196] T 7a66ceff1a2e452b96013c3b10c77faa P 280fe8782f3a420a917580684f455c7a: Deleting WAL directory at /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175691491353-13614-0/minicluster-data/ts-2/wal/wals/7a66ceff1a2e452b96013c3b10c77faa
I20260923 15:01:37.551271 15259 ts_tablet_manager.cc:1939] T 7a66ceff1a2e452b96013c3b10c77faa P 280fe8782f3a420a917580684f455c7a: Deleting consensus metadata
I20260923 15:01:37.551344 15236 tablet_bootstrap.cc:654] T 750be79c62374baea26e714cf95fae21 P 1cebda21ec684fcabfa5beaef4edda08: Neither blocks nor log segments found. Creating new log.
I20260923 15:01:37.551357 15240 tablet_bootstrap.cc:492] T 750be79c62374baea26e714cf95fae21 P 280fe8782f3a420a917580684f455c7a: Bootstrap starting.
I20260923 15:01:37.551791 15240 tablet_bootstrap.cc:654] T 750be79c62374baea26e714cf95fae21 P 280fe8782f3a420a917580684f455c7a: Neither blocks nor log segments found. Creating new log.
I20260923 15:01:37.552280 15236 tablet_bootstrap.cc:492] T 750be79c62374baea26e714cf95fae21 P 1cebda21ec684fcabfa5beaef4edda08: No bootstrap required, opened a new log
I20260923 15:01:37.552345 15236 ts_tablet_manager.cc:1397] T 750be79c62374baea26e714cf95fae21 P 1cebda21ec684fcabfa5beaef4edda08: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260923 15:01:37.552501 15236 raft_consensus.cc:348] T 750be79c62374baea26e714cf95fae21 P 1cebda21ec684fcabfa5beaef4edda08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cebda21ec684fcabfa5beaef4edda08" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 43089 } } peers { permanent_uuid: "d046a0f9cc58427986a4fa16815e54e6" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 35027 } } peers { permanent_uuid: "280fe8782f3a420a917580684f455c7a" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 34765 } }
I20260923 15:01:37.552583 15236 raft_consensus.cc:374] T 750be79c62374baea26e714cf95fae21 P 1cebda21ec684fcabfa5beaef4edda08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:01:37.552611 15236 raft_consensus.cc:729] T 750be79c62374baea26e714cf95fae21 P 1cebda21ec684fcabfa5beaef4edda08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1cebda21ec684fcabfa5beaef4edda08, State: Initialized, Role: FOLLOWER
I20260923 15:01:37.552651 15236 consensus_queue.cc:260] T 750be79c62374baea26e714cf95fae21 P 1cebda21ec684fcabfa5beaef4edda08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cebda21ec684fcabfa5beaef4edda08" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 43089 } } peers { permanent_uuid: "d046a0f9cc58427986a4fa16815e54e6" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 35027 } } peers { permanent_uuid: "280fe8782f3a420a917580684f455c7a" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 34765 } }
I20260923 15:01:37.552685 15237 tablet_bootstrap.cc:492] T 750be79c62374baea26e714cf95fae21 P d046a0f9cc58427986a4fa16815e54e6: No bootstrap required, opened a new log
I20260923 15:01:37.552726 15237 ts_tablet_manager.cc:1397] T 750be79c62374baea26e714cf95fae21 P d046a0f9cc58427986a4fa16815e54e6: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.000s
I20260923 15:01:37.552753 15236 ts_tablet_manager.cc:1428] T 750be79c62374baea26e714cf95fae21 P 1cebda21ec684fcabfa5beaef4edda08: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260923 15:01:37.552875 15237 raft_consensus.cc:348] T 750be79c62374baea26e714cf95fae21 P d046a0f9cc58427986a4fa16815e54e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cebda21ec684fcabfa5beaef4edda08" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 43089 } } peers { permanent_uuid: "d046a0f9cc58427986a4fa16815e54e6" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 35027 } } peers { permanent_uuid: "280fe8782f3a420a917580684f455c7a" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 34765 } }
I20260923 15:01:37.552265 15261 ts_tablet_manager.cc:1905] T 7a66ceff1a2e452b96013c3b10c77faa P 1cebda21ec684fcabfa5beaef4edda08: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 15:01:37.552948 15237 raft_consensus.cc:374] T 750be79c62374baea26e714cf95fae21 P d046a0f9cc58427986a4fa16815e54e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:01:37.552969 15237 raft_consensus.cc:729] T 750be79c62374baea26e714cf95fae21 P d046a0f9cc58427986a4fa16815e54e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d046a0f9cc58427986a4fa16815e54e6, State: Initialized, Role: FOLLOWER
I20260923 15:01:37.553009 15237 consensus_queue.cc:260] T 750be79c62374baea26e714cf95fae21 P d046a0f9cc58427986a4fa16815e54e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cebda21ec684fcabfa5beaef4edda08" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 43089 } } peers { permanent_uuid: "d046a0f9cc58427986a4fa16815e54e6" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 35027 } } peers { permanent_uuid: "280fe8782f3a420a917580684f455c7a" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 34765 } }
I20260923 15:01:37.553099 15237 ts_tablet_manager.cc:1428] T 750be79c62374baea26e714cf95fae21 P d046a0f9cc58427986a4fa16815e54e6: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260923 15:01:37.553251 15260 ts_tablet_manager.cc:1918] T 7a66ceff1a2e452b96013c3b10c77faa P d046a0f9cc58427986a4fa16815e54e6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 15:01:37.553308 15260 log.cc:1196] T 7a66ceff1a2e452b96013c3b10c77faa P d046a0f9cc58427986a4fa16815e54e6: Deleting WAL directory at /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175691491353-13614-0/minicluster-data/ts-1/wal/wals/7a66ceff1a2e452b96013c3b10c77faa
I20260923 15:01:37.553611 15260 ts_tablet_manager.cc:1939] T 7a66ceff1a2e452b96013c3b10c77faa P d046a0f9cc58427986a4fa16815e54e6: Deleting consensus metadata
I20260923 15:01:37.554122 14717 catalog_manager.cc:4841] TS d046a0f9cc58427986a4fa16815e54e6 (127.13.75.130:35027): tablet 7a66ceff1a2e452b96013c3b10c77faa (table test-table [id=5d365a222c2d44e8ace335d760ce93f7]) successfully deleted
I20260923 15:01:37.554279 14717 catalog_manager.cc:4841] TS 280fe8782f3a420a917580684f455c7a (127.13.75.131:34765): tablet 7a66ceff1a2e452b96013c3b10c77faa (table test-table [id=5d365a222c2d44e8ace335d760ce93f7]) successfully deleted
I20260923 15:01:37.554591 15261 ts_tablet_manager.cc:1918] T 7a66ceff1a2e452b96013c3b10c77faa P 1cebda21ec684fcabfa5beaef4edda08: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 15:01:37.554657 15261 log.cc:1196] T 7a66ceff1a2e452b96013c3b10c77faa P 1cebda21ec684fcabfa5beaef4edda08: Deleting WAL directory at /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175691491353-13614-0/minicluster-data/ts-0/wal/wals/7a66ceff1a2e452b96013c3b10c77faa
I20260923 15:01:37.554955 15261 ts_tablet_manager.cc:1939] T 7a66ceff1a2e452b96013c3b10c77faa P 1cebda21ec684fcabfa5beaef4edda08: Deleting consensus metadata
I20260923 15:01:37.555339 15240 tablet_bootstrap.cc:492] T 750be79c62374baea26e714cf95fae21 P 280fe8782f3a420a917580684f455c7a: No bootstrap required, opened a new log
I20260923 15:01:37.555395 15240 ts_tablet_manager.cc:1397] T 750be79c62374baea26e714cf95fae21 P 280fe8782f3a420a917580684f455c7a: Time spent bootstrapping tablet: real 0.004s user 0.001s sys 0.000s
I20260923 15:01:37.555402 14716 catalog_manager.cc:4841] TS 1cebda21ec684fcabfa5beaef4edda08 (127.13.75.129:43089): tablet 7a66ceff1a2e452b96013c3b10c77faa (table test-table [id=5d365a222c2d44e8ace335d760ce93f7]) successfully deleted
I20260923 15:01:37.555576 15240 raft_consensus.cc:348] T 750be79c62374baea26e714cf95fae21 P 280fe8782f3a420a917580684f455c7a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cebda21ec684fcabfa5beaef4edda08" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 43089 } } peers { permanent_uuid: "d046a0f9cc58427986a4fa16815e54e6" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 35027 } } peers { permanent_uuid: "280fe8782f3a420a917580684f455c7a" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 34765 } }
I20260923 15:01:37.555662 15240 raft_consensus.cc:374] T 750be79c62374baea26e714cf95fae21 P 280fe8782f3a420a917580684f455c7a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:01:37.555685 15240 raft_consensus.cc:729] T 750be79c62374baea26e714cf95fae21 P 280fe8782f3a420a917580684f455c7a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 280fe8782f3a420a917580684f455c7a, State: Initialized, Role: FOLLOWER
I20260923 15:01:37.555732 15240 consensus_queue.cc:260] T 750be79c62374baea26e714cf95fae21 P 280fe8782f3a420a917580684f455c7a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cebda21ec684fcabfa5beaef4edda08" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 43089 } } peers { permanent_uuid: "d046a0f9cc58427986a4fa16815e54e6" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 35027 } } peers { permanent_uuid: "280fe8782f3a420a917580684f455c7a" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 34765 } }
I20260923 15:01:37.555819 15240 ts_tablet_manager.cc:1428] T 750be79c62374baea26e714cf95fae21 P 280fe8782f3a420a917580684f455c7a: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260923 15:01:37.599160 15241 raft_consensus.cc:482] T 750be79c62374baea26e714cf95fae21 P 1cebda21ec684fcabfa5beaef4edda08 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 15:01:37.599270 15241 raft_consensus.cc:504] T 750be79c62374baea26e714cf95fae21 P 1cebda21ec684fcabfa5beaef4edda08 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cebda21ec684fcabfa5beaef4edda08" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 43089 } } peers { permanent_uuid: "d046a0f9cc58427986a4fa16815e54e6" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 35027 } } peers { permanent_uuid: "280fe8782f3a420a917580684f455c7a" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 34765 } }
I20260923 15:01:37.599598 15241 leader_election.cc:288] T 750be79c62374baea26e714cf95fae21 P 1cebda21ec684fcabfa5beaef4edda08 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d046a0f9cc58427986a4fa16815e54e6 (127.13.75.130:35027), 280fe8782f3a420a917580684f455c7a (127.13.75.131:34765)
W20260923 15:01:37.600600 15214 tablet.cc:2369] T 750be79c62374baea26e714cf95fae21 P 280fe8782f3a420a917580684f455c7a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 15:01:37.603724 15235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:37.599882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.129:47397 (local address 127.13.75.130:35027)
0923 15:01:37.599944 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0923 15:01:37.599946 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:01:37.600193 (+ 247us) server_negotiation.cc:407] Connection header received
0923 15:01:37.600394 (+ 201us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:01:37.600395 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:01:37.600403 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:01:37.600453 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:01:37.601296 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:37.602001 (+ 705us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:37.602994 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:37.603356 (+ 362us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:37.603439 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:01:37.603448 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0923 15:01:37.603605 (+ 157us) server_negotiation.cc:299] Negotiation successful
0923 15:01:37.603653 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260923 15:01:37.603722 15266 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:37.599778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.130:35027 (local address 127.13.75.129:47397)
0923 15:01:37.600116 (+ 338us) negotiation.cc:107] Waiting for socket to connect
0923 15:01:37.600128 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 15:01:37.600193 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:01:37.601139 (+ 946us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:01:37.601141 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:01:37.601143 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:01:37.601199 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:01:37.601200 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:01:37.602100 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:01:37.602101 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:01:37.602872 (+ 771us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:01:37.602875 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:01:37.603439 (+ 564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:01:37.603440 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:01:37.603515 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:01:37.603519 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 15:01:37.603578 (+ 59us) client_negotiation.cc:240] Negotiation successful
0923 15:01:37.603629 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":319,"thread_start_us":91,"threads_started":1}
I20260923 15:01:37.603751 15231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:37.600082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.129:49123 (local address 127.13.75.131:34765)
0923 15:01:37.600124 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0923 15:01:37.600125 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 15:01:37.600176 (+ 51us) server_negotiation.cc:407] Connection header received
0923 15:01:37.600275 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:01:37.600276 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:01:37.600287 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:01:37.600341 (+ 54us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:01:37.600480 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:37.601027 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:37.602331 (+ 1304us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:37.602919 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:37.602995 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:01:37.603005 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0923 15:01:37.603635 (+ 630us) server_negotiation.cc:299] Negotiation successful
0923 15:01:37.603680 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260923 15:01:37.603960 15027 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "750be79c62374baea26e714cf95fae21" candidate_uuid: "1cebda21ec684fcabfa5beaef4edda08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d046a0f9cc58427986a4fa16815e54e6" is_pre_election: true
I20260923 15:01:37.604061 15027 raft_consensus.cc:2454] T 750be79c62374baea26e714cf95fae21 P d046a0f9cc58427986a4fa16815e54e6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1cebda21ec684fcabfa5beaef4edda08 in term 0.
I20260923 15:01:37.604238 14821 leader_election.cc:302] T 750be79c62374baea26e714cf95fae21 P 1cebda21ec684fcabfa5beaef4edda08 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1cebda21ec684fcabfa5beaef4edda08, d046a0f9cc58427986a4fa16815e54e6; no voters:
I20260923 15:01:37.604400 15241 raft_consensus.cc:2781] T 750be79c62374baea26e714cf95fae21 P 1cebda21ec684fcabfa5beaef4edda08 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 15:01:37.604418 15265 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:37.600000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.131:34765 (local address 127.13.75.129:49123)
0923 15:01:37.600117 (+ 117us) negotiation.cc:107] Waiting for socket to connect
0923 15:01:37.600128 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 15:01:37.600193 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:01:37.600351 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:01:37.600352 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:01:37.600354 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:01:37.600404 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:01:37.600406 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:01:37.601115 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:01:37.601116 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:01:37.602236 (+ 1120us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:01:37.602238 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:01:37.603477 (+ 1239us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:01:37.603477 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:01:37.603539 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:01:37.603551 (+ 12us) client_negotiation.cc:769] Sending connection context
0923 15:01:37.604284 (+ 733us) client_negotiation.cc:240] Negotiation successful
0923 15:01:37.604336 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":71,"threads_started":1}
I20260923 15:01:37.604477 15241 raft_consensus.cc:482] T 750be79c62374baea26e714cf95fae21 P 1cebda21ec684fcabfa5beaef4edda08 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 15:01:37.604501 15241 raft_consensus.cc:3037] T 750be79c62374baea26e714cf95fae21 P 1cebda21ec684fcabfa5beaef4edda08 [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:01:37.604689 15167 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "750be79c62374baea26e714cf95fae21" candidate_uuid: "1cebda21ec684fcabfa5beaef4edda08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "280fe8782f3a420a917580684f455c7a" is_pre_election: true
I20260923 15:01:37.604827 15167 raft_consensus.cc:2454] T 750be79c62374baea26e714cf95fae21 P 280fe8782f3a420a917580684f455c7a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 1cebda21ec684fcabfa5beaef4edda08 in term 0.
I20260923 15:01:37.605396 15241 raft_consensus.cc:504] T 750be79c62374baea26e714cf95fae21 P 1cebda21ec684fcabfa5beaef4edda08 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cebda21ec684fcabfa5beaef4edda08" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 43089 } } peers { permanent_uuid: "d046a0f9cc58427986a4fa16815e54e6" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 35027 } } peers { permanent_uuid: "280fe8782f3a420a917580684f455c7a" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 34765 } }
I20260923 15:01:37.605531 15241 leader_election.cc:288] T 750be79c62374baea26e714cf95fae21 P 1cebda21ec684fcabfa5beaef4edda08 [CANDIDATE]: Term 1 election: Requested vote from peers d046a0f9cc58427986a4fa16815e54e6 (127.13.75.130:35027), 280fe8782f3a420a917580684f455c7a (127.13.75.131:34765)
I20260923 15:01:37.605746 15027 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "750be79c62374baea26e714cf95fae21" candidate_uuid: "1cebda21ec684fcabfa5beaef4edda08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d046a0f9cc58427986a4fa16815e54e6"
I20260923 15:01:37.605763 15167 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "750be79c62374baea26e714cf95fae21" candidate_uuid: "1cebda21ec684fcabfa5beaef4edda08" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "280fe8782f3a420a917580684f455c7a"
I20260923 15:01:37.605824 15027 raft_consensus.cc:3037] T 750be79c62374baea26e714cf95fae21 P d046a0f9cc58427986a4fa16815e54e6 [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:01:37.605846 15167 raft_consensus.cc:3037] T 750be79c62374baea26e714cf95fae21 P 280fe8782f3a420a917580684f455c7a [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:01:37.606590 15167 raft_consensus.cc:2454] T 750be79c62374baea26e714cf95fae21 P 280fe8782f3a420a917580684f455c7a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1cebda21ec684fcabfa5beaef4edda08 in term 1.
I20260923 15:01:37.606597 15027 raft_consensus.cc:2454] T 750be79c62374baea26e714cf95fae21 P d046a0f9cc58427986a4fa16815e54e6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 1cebda21ec684fcabfa5beaef4edda08 in term 1.
I20260923 15:01:37.606776 14821 leader_election.cc:302] T 750be79c62374baea26e714cf95fae21 P 1cebda21ec684fcabfa5beaef4edda08 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 1cebda21ec684fcabfa5beaef4edda08, d046a0f9cc58427986a4fa16815e54e6; no voters:
I20260923 15:01:37.606925 15241 raft_consensus.cc:2781] T 750be79c62374baea26e714cf95fae21 P 1cebda21ec684fcabfa5beaef4edda08 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 15:01:37.607065 15241 raft_consensus.cc:686] T 750be79c62374baea26e714cf95fae21 P 1cebda21ec684fcabfa5beaef4edda08 [term 1 LEADER]: Becoming Leader. State: Replica: 1cebda21ec684fcabfa5beaef4edda08, State: Running, Role: LEADER
I20260923 15:01:37.607132 15241 consensus_queue.cc:237] T 750be79c62374baea26e714cf95fae21 P 1cebda21ec684fcabfa5beaef4edda08 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cebda21ec684fcabfa5beaef4edda08" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 43089 } } peers { permanent_uuid: "d046a0f9cc58427986a4fa16815e54e6" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 35027 } } peers { permanent_uuid: "280fe8782f3a420a917580684f455c7a" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 34765 } }
I20260923 15:01:37.607841 14728 catalog_manager.cc:5495] T 750be79c62374baea26e714cf95fae21 P 1cebda21ec684fcabfa5beaef4edda08 reported cstate change: term changed from 0 to 1, leader changed from <none> to 1cebda21ec684fcabfa5beaef4edda08 (127.13.75.129). New cstate: current_term: 1 leader_uuid: "1cebda21ec684fcabfa5beaef4edda08" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cebda21ec684fcabfa5beaef4edda08" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 43089 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d046a0f9cc58427986a4fa16815e54e6" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 35027 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "280fe8782f3a420a917580684f455c7a" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 34765 } health_report { overall_health: UNKNOWN } } }
Sep 23 15:01:37 dist-test-slave-cnb0 krb5kdc[14685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175696, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.190@KRBTEST.COM for kudu/127.13.75.190@KRBTEST.COM
I20260923 15:01:37.643527 14781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:37.635711 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45310 (local address 127.13.75.190:35253)
0923 15:01:37.635763 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0923 15:01:37.635766 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:01:37.635903 (+ 137us) server_negotiation.cc:407] Connection header received
0923 15:01:37.635976 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:01:37.635977 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:01:37.635986 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:01:37.636040 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:01:37.636676 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:37.637173 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:37.638200 (+ 1027us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:37.638430 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:37.638490 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:01:37.640946 (+ 2456us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:01:37.640948 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:01:37.640948 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:01:37.640956 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:01:37.642351 (+ 1395us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:01:37.642736 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:01:37.642737 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:01:37.642737 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:01:37.642794 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:01:37.642992 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:01:37.642992 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:01:37.642993 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:01:37.643101 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:01:37.643173 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0923 15:01:37.643324 (+ 151us) server_negotiation.cc:299] Negotiation successful
0923 15:01:37.643411 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260923 15:01:37.643626 15274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:37.635625 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:35253 (local address 127.0.0.1:45310)
0923 15:01:37.635816 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0923 15:01:37.635829 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0923 15:01:37.635917 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:01:37.636071 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:01:37.636072 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:01:37.636491 (+ 419us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:01:37.636566 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:01:37.636570 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:01:37.637266 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:01:37.637267 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:01:37.638077 (+ 810us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:01:37.638081 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:01:37.638504 (+ 423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:01:37.638505 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:01:37.638565 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:01:37.639207 (+ 642us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:01:37.639213 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:01:37.640857 (+ 1644us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:01:37.642449 (+ 1592us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:01:37.642458 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:01:37.642461 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:01:37.642661 (+ 200us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:01:37.642857 (+ 196us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:01:37.642858 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:01:37.642859 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:01:37.642934 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:01:37.643164 (+ 230us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:01:37.643173 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:01:37.643221 (+ 48us) client_negotiation.cc:769] Sending connection context
0923 15:01:37.643303 (+ 82us) client_negotiation.cc:240] Negotiation successful
0923 15:01:37.643481 (+ 178us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"mutex_wait_us":73,"thread_start_us":96,"threads_started":1}
I20260923 15:01:37.645744 14728 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.13.75.190@KRBTEST.COM'} at 127.0.0.1:45310:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260923 15:01:37.645968 14728 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 15:01:37.647518 15007 tablet_service.cc:1467] Processing CreateTablet for tablet 10040658fd2d42c285cbe182ef2b318b (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=be4294d313b546948a2b36e2129f6f14]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260923 15:01:37.647522 14867 tablet_service.cc:1467] Processing CreateTablet for tablet 10040658fd2d42c285cbe182ef2b318b (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=be4294d313b546948a2b36e2129f6f14]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260923 15:01:37.647692 14867 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 10040658fd2d42c285cbe182ef2b318b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:01:37.647693 15007 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 10040658fd2d42c285cbe182ef2b318b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:01:37.649154 15237 tablet_bootstrap.cc:492] T 10040658fd2d42c285cbe182ef2b318b P d046a0f9cc58427986a4fa16815e54e6: Bootstrap starting.
I20260923 15:01:37.649263 15147 tablet_service.cc:1467] Processing CreateTablet for tablet 10040658fd2d42c285cbe182ef2b318b (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=be4294d313b546948a2b36e2129f6f14]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260923 15:01:37.649317 15236 tablet_bootstrap.cc:492] T 10040658fd2d42c285cbe182ef2b318b P 1cebda21ec684fcabfa5beaef4edda08: Bootstrap starting.
I20260923 15:01:37.649402 15147 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 10040658fd2d42c285cbe182ef2b318b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:01:37.649650 15237 tablet_bootstrap.cc:654] T 10040658fd2d42c285cbe182ef2b318b P d046a0f9cc58427986a4fa16815e54e6: Neither blocks nor log segments found. Creating new log.
I20260923 15:01:37.649724 15236 tablet_bootstrap.cc:654] T 10040658fd2d42c285cbe182ef2b318b P 1cebda21ec684fcabfa5beaef4edda08: Neither blocks nor log segments found. Creating new log.
I20260923 15:01:37.650454 15237 tablet_bootstrap.cc:492] T 10040658fd2d42c285cbe182ef2b318b P d046a0f9cc58427986a4fa16815e54e6: No bootstrap required, opened a new log
I20260923 15:01:37.650516 15237 ts_tablet_manager.cc:1397] T 10040658fd2d42c285cbe182ef2b318b P d046a0f9cc58427986a4fa16815e54e6: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.001s
I20260923 15:01:37.650615 15236 tablet_bootstrap.cc:492] T 10040658fd2d42c285cbe182ef2b318b P 1cebda21ec684fcabfa5beaef4edda08: No bootstrap required, opened a new log
I20260923 15:01:37.650656 15236 ts_tablet_manager.cc:1397] T 10040658fd2d42c285cbe182ef2b318b P 1cebda21ec684fcabfa5beaef4edda08: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260923 15:01:37.650668 15237 raft_consensus.cc:348] T 10040658fd2d42c285cbe182ef2b318b P d046a0f9cc58427986a4fa16815e54e6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cebda21ec684fcabfa5beaef4edda08" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 43089 } } peers { permanent_uuid: "d046a0f9cc58427986a4fa16815e54e6" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 35027 } } peers { permanent_uuid: "280fe8782f3a420a917580684f455c7a" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 34765 } }
I20260923 15:01:37.650741 15237 raft_consensus.cc:374] T 10040658fd2d42c285cbe182ef2b318b P d046a0f9cc58427986a4fa16815e54e6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:01:37.650761 15237 raft_consensus.cc:729] T 10040658fd2d42c285cbe182ef2b318b P d046a0f9cc58427986a4fa16815e54e6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d046a0f9cc58427986a4fa16815e54e6, State: Initialized, Role: FOLLOWER
I20260923 15:01:37.650803 15237 consensus_queue.cc:260] T 10040658fd2d42c285cbe182ef2b318b P d046a0f9cc58427986a4fa16815e54e6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cebda21ec684fcabfa5beaef4edda08" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 43089 } } peers { permanent_uuid: "d046a0f9cc58427986a4fa16815e54e6" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 35027 } } peers { permanent_uuid: "280fe8782f3a420a917580684f455c7a" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 34765 } }
I20260923 15:01:37.650789 15236 raft_consensus.cc:348] T 10040658fd2d42c285cbe182ef2b318b P 1cebda21ec684fcabfa5beaef4edda08 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cebda21ec684fcabfa5beaef4edda08" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 43089 } } peers { permanent_uuid: "d046a0f9cc58427986a4fa16815e54e6" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 35027 } } peers { permanent_uuid: "280fe8782f3a420a917580684f455c7a" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 34765 } }
I20260923 15:01:37.650851 15236 raft_consensus.cc:374] T 10040658fd2d42c285cbe182ef2b318b P 1cebda21ec684fcabfa5beaef4edda08 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:01:37.650871 15236 raft_consensus.cc:729] T 10040658fd2d42c285cbe182ef2b318b P 1cebda21ec684fcabfa5beaef4edda08 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1cebda21ec684fcabfa5beaef4edda08, State: Initialized, Role: FOLLOWER
I20260923 15:01:37.650915 15237 ts_tablet_manager.cc:1428] T 10040658fd2d42c285cbe182ef2b318b P d046a0f9cc58427986a4fa16815e54e6: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260923 15:01:37.650947 15236 consensus_queue.cc:260] T 10040658fd2d42c285cbe182ef2b318b P 1cebda21ec684fcabfa5beaef4edda08 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cebda21ec684fcabfa5beaef4edda08" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 43089 } } peers { permanent_uuid: "d046a0f9cc58427986a4fa16815e54e6" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 35027 } } peers { permanent_uuid: "280fe8782f3a420a917580684f455c7a" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 34765 } }
I20260923 15:01:37.650974 15240 tablet_bootstrap.cc:492] T 10040658fd2d42c285cbe182ef2b318b P 280fe8782f3a420a917580684f455c7a: Bootstrap starting.
I20260923 15:01:37.651048 15236 ts_tablet_manager.cc:1428] T 10040658fd2d42c285cbe182ef2b318b P 1cebda21ec684fcabfa5beaef4edda08: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260923 15:01:37.651165 15243 tablet_replica.cc:439] T 10040658fd2d42c285cbe182ef2b318b P d046a0f9cc58427986a4fa16815e54e6: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cebda21ec684fcabfa5beaef4edda08" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 43089 } } peers { permanent_uuid: "d046a0f9cc58427986a4fa16815e54e6" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 35027 } } peers { permanent_uuid: "280fe8782f3a420a917580684f455c7a" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 34765 } } }
I20260923 15:01:37.651264 15243 tablet_replica.cc:442] T 10040658fd2d42c285cbe182ef2b318b P d046a0f9cc58427986a4fa16815e54e6: This TxnStatusTablet replica's current role is: FOLLOWER
I20260923 15:01:37.651433 15240 tablet_bootstrap.cc:654] T 10040658fd2d42c285cbe182ef2b318b P 280fe8782f3a420a917580684f455c7a: Neither blocks nor log segments found. Creating new log.
I20260923 15:01:37.651865 15241 tablet_replica.cc:439] T 10040658fd2d42c285cbe182ef2b318b P 1cebda21ec684fcabfa5beaef4edda08: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cebda21ec684fcabfa5beaef4edda08" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 43089 } } peers { permanent_uuid: "d046a0f9cc58427986a4fa16815e54e6" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 35027 } } peers { permanent_uuid: "280fe8782f3a420a917580684f455c7a" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 34765 } } }
I20260923 15:01:37.651969 15241 tablet_replica.cc:442] T 10040658fd2d42c285cbe182ef2b318b P 1cebda21ec684fcabfa5beaef4edda08: This TxnStatusTablet replica's current role is: FOLLOWER
I20260923 15:01:37.652675 15240 tablet_bootstrap.cc:492] T 10040658fd2d42c285cbe182ef2b318b P 280fe8782f3a420a917580684f455c7a: No bootstrap required, opened a new log
I20260923 15:01:37.652730 15240 ts_tablet_manager.cc:1397] T 10040658fd2d42c285cbe182ef2b318b P 280fe8782f3a420a917580684f455c7a: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 15:01:37.652882 15240 raft_consensus.cc:348] T 10040658fd2d42c285cbe182ef2b318b P 280fe8782f3a420a917580684f455c7a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cebda21ec684fcabfa5beaef4edda08" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 43089 } } peers { permanent_uuid: "d046a0f9cc58427986a4fa16815e54e6" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 35027 } } peers { permanent_uuid: "280fe8782f3a420a917580684f455c7a" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 34765 } }
I20260923 15:01:37.652967 15240 raft_consensus.cc:374] T 10040658fd2d42c285cbe182ef2b318b P 280fe8782f3a420a917580684f455c7a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:01:37.652990 15240 raft_consensus.cc:729] T 10040658fd2d42c285cbe182ef2b318b P 280fe8782f3a420a917580684f455c7a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 280fe8782f3a420a917580684f455c7a, State: Initialized, Role: FOLLOWER
I20260923 15:01:37.653039 15240 consensus_queue.cc:260] T 10040658fd2d42c285cbe182ef2b318b P 280fe8782f3a420a917580684f455c7a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cebda21ec684fcabfa5beaef4edda08" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 43089 } } peers { permanent_uuid: "d046a0f9cc58427986a4fa16815e54e6" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 35027 } } peers { permanent_uuid: "280fe8782f3a420a917580684f455c7a" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 34765 } }
I20260923 15:01:37.653138 15240 ts_tablet_manager.cc:1428] T 10040658fd2d42c285cbe182ef2b318b P 280fe8782f3a420a917580684f455c7a: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260923 15:01:37.653173 15246 tablet_replica.cc:439] T 10040658fd2d42c285cbe182ef2b318b P 280fe8782f3a420a917580684f455c7a: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cebda21ec684fcabfa5beaef4edda08" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 43089 } } peers { permanent_uuid: "d046a0f9cc58427986a4fa16815e54e6" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 35027 } } peers { permanent_uuid: "280fe8782f3a420a917580684f455c7a" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 34765 } } }
I20260923 15:01:37.653273 15246 tablet_replica.cc:442] T 10040658fd2d42c285cbe182ef2b318b P 280fe8782f3a420a917580684f455c7a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260923 15:01:37.762676 15243 raft_consensus.cc:482] T 10040658fd2d42c285cbe182ef2b318b P d046a0f9cc58427986a4fa16815e54e6 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 15:01:37.762791 15243 raft_consensus.cc:504] T 10040658fd2d42c285cbe182ef2b318b P d046a0f9cc58427986a4fa16815e54e6 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cebda21ec684fcabfa5beaef4edda08" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 43089 } } peers { permanent_uuid: "d046a0f9cc58427986a4fa16815e54e6" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 35027 } } peers { permanent_uuid: "280fe8782f3a420a917580684f455c7a" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 34765 } }
I20260923 15:01:37.763258 15243 leader_election.cc:288] T 10040658fd2d42c285cbe182ef2b318b P d046a0f9cc58427986a4fa16815e54e6 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1cebda21ec684fcabfa5beaef4edda08 (127.13.75.129:43089), 280fe8782f3a420a917580684f455c7a (127.13.75.131:34765)
I20260923 15:01:37.767585 15234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:37.763631 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.130:55369 (local address 127.13.75.129:43089)
0923 15:01:37.763685 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0923 15:01:37.763687 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:01:37.763730 (+ 43us) server_negotiation.cc:407] Connection header received
0923 15:01:37.763820 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:01:37.763821 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:01:37.763832 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:01:37.763892 (+ 60us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:01:37.764108 (+ 216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:37.764860 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:37.766551 (+ 1691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:37.767136 (+ 585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:37.767217 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:01:37.767226 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0923 15:01:37.767455 (+ 229us) server_negotiation.cc:299] Negotiation successful
0923 15:01:37.767509 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260923 15:01:37.767612 15231 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:37.763539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.130:55671 (local address 127.13.75.131:34765)
0923 15:01:37.763594 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0923 15:01:37.763597 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:01:37.763745 (+ 148us) server_negotiation.cc:407] Connection header received
0923 15:01:37.763820 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:01:37.763821 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:01:37.763835 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:01:37.763892 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:01:37.764108 (+ 216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:37.764870 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:37.766574 (+ 1704us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:37.767137 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:37.767217 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:01:37.767226 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0923 15:01:37.767454 (+ 228us) server_negotiation.cc:299] Negotiation successful
0923 15:01:37.767551 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260923 15:01:37.767858 15279 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:37.763543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.129:43089 (local address 127.13.75.130:55369)
0923 15:01:37.763658 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0923 15:01:37.763671 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0923 15:01:37.763741 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:01:37.763926 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:01:37.763927 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:01:37.763930 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:01:37.764033 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:01:37.764034 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:01:37.764975 (+ 941us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:01:37.764976 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:01:37.766427 (+ 1451us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:01:37.766431 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:01:37.767248 (+ 817us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:01:37.767249 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:01:37.767348 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:01:37.767352 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 15:01:37.767693 (+ 341us) client_negotiation.cc:240] Negotiation successful
0923 15:01:37.767774 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":100,"mutex_wait_us":59,"thread_start_us":53,"threads_started":1}
I20260923 15:01:37.767858 15278 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:37.763471 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.131:34765 (local address 127.13.75.130:55671)
0923 15:01:37.763636 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0923 15:01:37.763648 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 15:01:37.763722 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:01:37.763928 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:01:37.763928 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:01:37.763936 (+ 8us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:01:37.764019 (+ 83us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:01:37.764021 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:01:37.764975 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:01:37.764976 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:01:37.766480 (+ 1504us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:01:37.766482 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:01:37.767249 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:01:37.767250 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:01:37.767332 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:01:37.767338 (+ 6us) client_negotiation.cc:769] Sending connection context
0923 15:01:37.767693 (+ 355us) client_negotiation.cc:240] Negotiation successful
0923 15:01:37.767760 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"mutex_wait_us":118,"thread_start_us":82,"threads_started":1}
I20260923 15:01:37.768141 15167 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "10040658fd2d42c285cbe182ef2b318b" candidate_uuid: "d046a0f9cc58427986a4fa16815e54e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "280fe8782f3a420a917580684f455c7a" is_pre_election: true
I20260923 15:01:37.768270 15167 raft_consensus.cc:2454] T 10040658fd2d42c285cbe182ef2b318b P 280fe8782f3a420a917580684f455c7a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d046a0f9cc58427986a4fa16815e54e6 in term 0.
I20260923 15:01:37.768237 14887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "10040658fd2d42c285cbe182ef2b318b" candidate_uuid: "d046a0f9cc58427986a4fa16815e54e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1cebda21ec684fcabfa5beaef4edda08" is_pre_election: true
I20260923 15:01:37.768334 14887 raft_consensus.cc:2454] T 10040658fd2d42c285cbe182ef2b318b P 1cebda21ec684fcabfa5beaef4edda08 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate d046a0f9cc58427986a4fa16815e54e6 in term 0.
I20260923 15:01:37.768473 14961 leader_election.cc:302] T 10040658fd2d42c285cbe182ef2b318b P d046a0f9cc58427986a4fa16815e54e6 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 280fe8782f3a420a917580684f455c7a, d046a0f9cc58427986a4fa16815e54e6; no voters:
I20260923 15:01:37.768628 15243 raft_consensus.cc:2781] T 10040658fd2d42c285cbe182ef2b318b P d046a0f9cc58427986a4fa16815e54e6 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 15:01:37.768672 15243 raft_consensus.cc:482] T 10040658fd2d42c285cbe182ef2b318b P d046a0f9cc58427986a4fa16815e54e6 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 15:01:37.768697 15243 raft_consensus.cc:3037] T 10040658fd2d42c285cbe182ef2b318b P d046a0f9cc58427986a4fa16815e54e6 [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:01:37.769397 15243 raft_consensus.cc:504] T 10040658fd2d42c285cbe182ef2b318b P d046a0f9cc58427986a4fa16815e54e6 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cebda21ec684fcabfa5beaef4edda08" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 43089 } } peers { permanent_uuid: "d046a0f9cc58427986a4fa16815e54e6" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 35027 } } peers { permanent_uuid: "280fe8782f3a420a917580684f455c7a" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 34765 } }
I20260923 15:01:37.769537 15243 leader_election.cc:288] T 10040658fd2d42c285cbe182ef2b318b P d046a0f9cc58427986a4fa16815e54e6 [CANDIDATE]: Term 1 election: Requested vote from peers 1cebda21ec684fcabfa5beaef4edda08 (127.13.75.129:43089), 280fe8782f3a420a917580684f455c7a (127.13.75.131:34765)
I20260923 15:01:37.769731 14887 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "10040658fd2d42c285cbe182ef2b318b" candidate_uuid: "d046a0f9cc58427986a4fa16815e54e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1cebda21ec684fcabfa5beaef4edda08"
I20260923 15:01:37.769717 15167 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "10040658fd2d42c285cbe182ef2b318b" candidate_uuid: "d046a0f9cc58427986a4fa16815e54e6" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "280fe8782f3a420a917580684f455c7a"
I20260923 15:01:37.769805 14887 raft_consensus.cc:3037] T 10040658fd2d42c285cbe182ef2b318b P 1cebda21ec684fcabfa5beaef4edda08 [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:01:37.769805 15167 raft_consensus.cc:3037] T 10040658fd2d42c285cbe182ef2b318b P 280fe8782f3a420a917580684f455c7a [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:01:37.770596 15167 raft_consensus.cc:2454] T 10040658fd2d42c285cbe182ef2b318b P 280fe8782f3a420a917580684f455c7a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d046a0f9cc58427986a4fa16815e54e6 in term 1.
I20260923 15:01:37.770596 14887 raft_consensus.cc:2454] T 10040658fd2d42c285cbe182ef2b318b P 1cebda21ec684fcabfa5beaef4edda08 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate d046a0f9cc58427986a4fa16815e54e6 in term 1.
I20260923 15:01:37.770768 14961 leader_election.cc:302] T 10040658fd2d42c285cbe182ef2b318b P d046a0f9cc58427986a4fa16815e54e6 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 280fe8782f3a420a917580684f455c7a, d046a0f9cc58427986a4fa16815e54e6; no voters:
I20260923 15:01:37.770919 15243 raft_consensus.cc:2781] T 10040658fd2d42c285cbe182ef2b318b P d046a0f9cc58427986a4fa16815e54e6 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 15:01:37.771056 15243 raft_consensus.cc:686] T 10040658fd2d42c285cbe182ef2b318b P d046a0f9cc58427986a4fa16815e54e6 [term 1 LEADER]: Becoming Leader. State: Replica: d046a0f9cc58427986a4fa16815e54e6, State: Running, Role: LEADER
I20260923 15:01:37.771155 15243 consensus_queue.cc:237] T 10040658fd2d42c285cbe182ef2b318b P d046a0f9cc58427986a4fa16815e54e6 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cebda21ec684fcabfa5beaef4edda08" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 43089 } } peers { permanent_uuid: "d046a0f9cc58427986a4fa16815e54e6" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 35027 } } peers { permanent_uuid: "280fe8782f3a420a917580684f455c7a" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 34765 } }
I20260923 15:01:37.771525 15280 tablet_replica.cc:439] T 10040658fd2d42c285cbe182ef2b318b P d046a0f9cc58427986a4fa16815e54e6: TxnStatusTablet state changed. Reason: New leader d046a0f9cc58427986a4fa16815e54e6. Latest consensus state: current_term: 1 leader_uuid: "d046a0f9cc58427986a4fa16815e54e6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cebda21ec684fcabfa5beaef4edda08" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 43089 } } peers { permanent_uuid: "d046a0f9cc58427986a4fa16815e54e6" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 35027 } } peers { permanent_uuid: "280fe8782f3a420a917580684f455c7a" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 34765 } } }
I20260923 15:01:37.771613 15280 tablet_replica.cc:442] T 10040658fd2d42c285cbe182ef2b318b P d046a0f9cc58427986a4fa16815e54e6: This TxnStatusTablet replica's current role is: LEADER
I20260923 15:01:37.771804 15282 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260923 15:01:37.771929 15282 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260923 15:01:37.771873 14729 catalog_manager.cc:5495] T 10040658fd2d42c285cbe182ef2b318b P d046a0f9cc58427986a4fa16815e54e6 reported cstate change: term changed from 0 to 1, leader changed from <none> to d046a0f9cc58427986a4fa16815e54e6 (127.13.75.130). New cstate: current_term: 1 leader_uuid: "d046a0f9cc58427986a4fa16815e54e6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cebda21ec684fcabfa5beaef4edda08" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 43089 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "d046a0f9cc58427986a4fa16815e54e6" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 35027 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "280fe8782f3a420a917580684f455c7a" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 34765 } health_report { overall_health: UNKNOWN } } }
W20260923 15:01:37.795281 14934 tablet.cc:2369] T 10040658fd2d42c285cbe182ef2b318b P 1cebda21ec684fcabfa5beaef4edda08: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 15:01:37.802304 15235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:37.799756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49896 (local address 127.13.75.130:35027)
0923 15:01:37.799814 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0923 15:01:37.799816 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:01:37.799825 (+ 9us) server_negotiation.cc:407] Connection header received
0923 15:01:37.799868 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:01:37.799869 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:01:37.799886 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:01:37.799956 (+ 70us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 15:01:37.800120 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:37.800593 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:37.801397 (+ 804us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:37.801611 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:37.801681 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:01:37.801898 (+ 217us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 15:01:37.802012 (+ 114us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 15:01:37.802089 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0923 15:01:37.802183 (+ 94us) server_negotiation.cc:299] Negotiation successful
0923 15:01:37.802206 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260923 15:01:37.802307 15274 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:37.799507 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.130:35027 (local address 127.0.0.1:49896)
0923 15:01:37.799597 (+ 90us) negotiation.cc:107] Waiting for socket to connect
0923 15:01:37.799608 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 15:01:37.799657 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:01:37.799964 (+ 307us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:01:37.799965 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:01:37.799966 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0923 15:01:37.800034 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:01:37.800037 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:01:37.800671 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:01:37.800672 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:01:37.801310 (+ 638us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:01:37.801312 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:01:37.801692 (+ 380us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:01:37.801693 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:01:37.801776 (+ 83us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:01:37.801791 (+ 15us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0923 15:01:37.802083 (+ 292us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0923 15:01:37.802094 (+ 11us) client_negotiation.cc:769] Sending connection context
0923 15:01:37.802181 (+ 87us) client_negotiation.cc:240] Negotiation successful
0923 15:01:37.802234 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48}
I20260923 15:01:37.803606 14887 raft_consensus.cc:1261] T 10040658fd2d42c285cbe182ef2b318b P 1cebda21ec684fcabfa5beaef4edda08 [term 1 FOLLOWER]: Refusing update from remote peer d046a0f9cc58427986a4fa16815e54e6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 15:01:37.803753 15167 raft_consensus.cc:1261] T 10040658fd2d42c285cbe182ef2b318b P 280fe8782f3a420a917580684f455c7a [term 1 FOLLOWER]: Refusing update from remote peer d046a0f9cc58427986a4fa16815e54e6: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 15:01:37.803949 15243 consensus_queue.cc:1035] T 10040658fd2d42c285cbe182ef2b318b P d046a0f9cc58427986a4fa16815e54e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1cebda21ec684fcabfa5beaef4edda08" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 43089 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 15:01:37.804091 15280 consensus_queue.cc:1035] T 10040658fd2d42c285cbe182ef2b318b P d046a0f9cc58427986a4fa16815e54e6 [LEADER]: Connected to new peer: Peer: permanent_uuid: "280fe8782f3a420a917580684f455c7a" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 34765 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 15:01:37.804684 15241 tablet_replica.cc:439] T 10040658fd2d42c285cbe182ef2b318b P 1cebda21ec684fcabfa5beaef4edda08: TxnStatusTablet state changed. Reason: New leader d046a0f9cc58427986a4fa16815e54e6. Latest consensus state: current_term: 1 leader_uuid: "d046a0f9cc58427986a4fa16815e54e6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cebda21ec684fcabfa5beaef4edda08" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 43089 } } peers { permanent_uuid: "d046a0f9cc58427986a4fa16815e54e6" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 35027 } } peers { permanent_uuid: "280fe8782f3a420a917580684f455c7a" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 34765 } } }
I20260923 15:01:37.804687 15246 tablet_replica.cc:439] T 10040658fd2d42c285cbe182ef2b318b P 280fe8782f3a420a917580684f455c7a: TxnStatusTablet state changed. Reason: New leader d046a0f9cc58427986a4fa16815e54e6. Latest consensus state: current_term: 1 leader_uuid: "d046a0f9cc58427986a4fa16815e54e6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cebda21ec684fcabfa5beaef4edda08" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 43089 } } peers { permanent_uuid: "d046a0f9cc58427986a4fa16815e54e6" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 35027 } } peers { permanent_uuid: "280fe8782f3a420a917580684f455c7a" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 34765 } } }
I20260923 15:01:37.804778 15241 tablet_replica.cc:442] T 10040658fd2d42c285cbe182ef2b318b P 1cebda21ec684fcabfa5beaef4edda08: This TxnStatusTablet replica's current role is: FOLLOWER
I20260923 15:01:37.804780 15246 tablet_replica.cc:442] T 10040658fd2d42c285cbe182ef2b318b P 280fe8782f3a420a917580684f455c7a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260923 15:01:37.805358 15280 tablet_replica.cc:439] T 10040658fd2d42c285cbe182ef2b318b P d046a0f9cc58427986a4fa16815e54e6: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d046a0f9cc58427986a4fa16815e54e6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cebda21ec684fcabfa5beaef4edda08" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 43089 } } peers { permanent_uuid: "d046a0f9cc58427986a4fa16815e54e6" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 35027 } } peers { permanent_uuid: "280fe8782f3a420a917580684f455c7a" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 34765 } } }
I20260923 15:01:37.805430 15280 tablet_replica.cc:442] T 10040658fd2d42c285cbe182ef2b318b P d046a0f9cc58427986a4fa16815e54e6: This TxnStatusTablet replica's current role is: LEADER
I20260923 15:01:37.805843 15280 tablet_replica.cc:439] T 10040658fd2d42c285cbe182ef2b318b P d046a0f9cc58427986a4fa16815e54e6: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "d046a0f9cc58427986a4fa16815e54e6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cebda21ec684fcabfa5beaef4edda08" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 43089 } } peers { permanent_uuid: "d046a0f9cc58427986a4fa16815e54e6" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 35027 } } peers { permanent_uuid: "280fe8782f3a420a917580684f455c7a" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 34765 } } }
I20260923 15:01:37.805950 15280 tablet_replica.cc:442] T 10040658fd2d42c285cbe182ef2b318b P d046a0f9cc58427986a4fa16815e54e6: This TxnStatusTablet replica's current role is: LEADER
I20260923 15:01:37.806351 15241 tablet_replica.cc:439] T 10040658fd2d42c285cbe182ef2b318b P 1cebda21ec684fcabfa5beaef4edda08: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d046a0f9cc58427986a4fa16815e54e6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cebda21ec684fcabfa5beaef4edda08" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 43089 } } peers { permanent_uuid: "d046a0f9cc58427986a4fa16815e54e6" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 35027 } } peers { permanent_uuid: "280fe8782f3a420a917580684f455c7a" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 34765 } } }
I20260923 15:01:37.806435 15241 tablet_replica.cc:442] T 10040658fd2d42c285cbe182ef2b318b P 1cebda21ec684fcabfa5beaef4edda08: This TxnStatusTablet replica's current role is: FOLLOWER
I20260923 15:01:37.806931 15246 tablet_replica.cc:439] T 10040658fd2d42c285cbe182ef2b318b P 280fe8782f3a420a917580684f455c7a: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "d046a0f9cc58427986a4fa16815e54e6" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1cebda21ec684fcabfa5beaef4edda08" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 43089 } } peers { permanent_uuid: "d046a0f9cc58427986a4fa16815e54e6" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 35027 } } peers { permanent_uuid: "280fe8782f3a420a917580684f455c7a" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 34765 } } }
I20260923 15:01:37.807016 15246 tablet_replica.cc:442] T 10040658fd2d42c285cbe182ef2b318b P 280fe8782f3a420a917580684f455c7a: This TxnStatusTablet replica's current role is: FOLLOWER
I20260923 15:01:37.810424 15234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:37.807792 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52242 (local address 127.13.75.129:43089)
0923 15:01:37.807837 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0923 15:01:37.807839 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:01:37.807893 (+ 54us) server_negotiation.cc:407] Connection header received
0923 15:01:37.807985 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:01:37.807986 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:01:37.808005 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:01:37.808061 (+ 56us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 15:01:37.808185 (+ 124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:37.808631 (+ 446us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:37.809704 (+ 1073us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:37.809891 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:37.809953 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:01:37.810111 (+ 158us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 15:01:37.810205 (+ 94us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 15:01:37.810258 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0923 15:01:37.810321 (+ 63us) server_negotiation.cc:299] Negotiation successful
0923 15:01:37.810340 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260923 15:01:37.816141 15235 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:37.813295 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.129:43657 (local address 127.13.75.130:35027)
0923 15:01:37.813364 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0923 15:01:37.813366 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:01:37.813543 (+ 177us) server_negotiation.cc:407] Connection header received
0923 15:01:37.813576 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:01:37.813577 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:01:37.813589 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:01:37.813648 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 15:01:37.813831 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:37.814302 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:37.815469 (+ 1167us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:37.815671 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:37.815731 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:01:37.815931 (+ 200us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 15:01:37.815958 (+ 27us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 15:01:37.816002 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0923 15:01:37.816064 (+ 62us) server_negotiation.cc:299] Negotiation successful
0923 15:01:37.816083 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":30}
I20260923 15:01:37.816205 15289 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:37.813197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.130:35027 (local address 127.13.75.129:43657)
0923 15:01:37.813410 (+ 213us) negotiation.cc:107] Waiting for socket to connect
0923 15:01:37.813421 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 15:01:37.813480 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:01:37.813664 (+ 184us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:01:37.813665 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:01:37.813667 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0923 15:01:37.813732 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:01:37.813734 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:01:37.814377 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:01:37.814378 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:01:37.815366 (+ 988us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:01:37.815369 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:01:37.815762 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:01:37.815763 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:01:37.815836 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:01:37.815853 (+ 17us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0923 15:01:37.816011 (+ 158us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0923 15:01:37.816012 (+ 1us) client_negotiation.cc:769] Sending connection context
0923 15:01:37.816061 (+ 49us) client_negotiation.cc:240] Negotiation successful
0923 15:01:37.816105 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166,"thread_start_us":87,"threads_started":1}
I20260923 15:01:37.818485 15167 raft_consensus.cc:1261] T 750be79c62374baea26e714cf95fae21 P 280fe8782f3a420a917580684f455c7a [term 1 FOLLOWER]: Refusing update from remote peer 1cebda21ec684fcabfa5beaef4edda08: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 15:01:37.818518 15027 raft_consensus.cc:1261] T 750be79c62374baea26e714cf95fae21 P d046a0f9cc58427986a4fa16815e54e6 [term 1 FOLLOWER]: Refusing update from remote peer 1cebda21ec684fcabfa5beaef4edda08: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 15:01:37.818818 15241 consensus_queue.cc:1035] T 750be79c62374baea26e714cf95fae21 P 1cebda21ec684fcabfa5beaef4edda08 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d046a0f9cc58427986a4fa16815e54e6" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 35027 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 15:01:37.818996 15267 consensus_queue.cc:1035] T 750be79c62374baea26e714cf95fae21 P 1cebda21ec684fcabfa5beaef4edda08 [LEADER]: Connected to new peer: Peer: permanent_uuid: "280fe8782f3a420a917580684f455c7a" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 34765 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 15:01:37.830505 15295 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:37.827318 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.129:43089 (local address 127.13.75.130:35015)
0923 15:01:37.827561 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0923 15:01:37.827588 (+ 27us) client_negotiation.cc:174] Beginning negotiation
0923 15:01:37.827653 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:01:37.827866 (+ 213us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:01:37.827867 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:01:37.827868 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0923 15:01:37.827948 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:01:37.827950 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:01:37.828552 (+ 602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:01:37.828553 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:01:37.829497 (+ 944us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:01:37.829500 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:01:37.830039 (+ 539us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:01:37.830040 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:01:37.830106 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:01:37.830127 (+ 21us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0923 15:01:37.830305 (+ 178us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0923 15:01:37.830306 (+ 1us) client_negotiation.cc:769] Sending connection context
0923 15:01:37.830350 (+ 44us) client_negotiation.cc:240] Negotiation successful
0923 15:01:37.830402 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":80,"threads_started":1}
I20260923 15:01:37.830533 15234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:37.827346 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.130:35015 (local address 127.13.75.129:43089)
0923 15:01:37.827437 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0923 15:01:37.827439 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:01:37.827716 (+ 277us) server_negotiation.cc:407] Connection header received
0923 15:01:37.827744 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:01:37.827744 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:01:37.827756 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:01:37.827809 (+ 53us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 15:01:37.828008 (+ 199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:37.828478 (+ 470us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:37.829603 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:37.829923 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:37.829964 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:01:37.830209 (+ 245us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 15:01:37.830245 (+ 36us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 15:01:37.830270 (+ 25us) server_negotiation.cc:1035] Waiting for connection context
0923 15:01:37.830428 (+ 158us) server_negotiation.cc:299] Negotiation successful
0923 15:01:37.830456 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260923 15:01:37.842710 14729 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45296:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 15:01:37.842841 14729 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45296:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 15:01:37.844758 14729 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 61f4472f8f444f179d09ce5ce961925c: Sending DeleteTablet for 3 replicas of tablet 750be79c62374baea26e714cf95fae21
I20260923 15:01:37.845094 14867 tablet_service.cc:1514] Processing DeleteTablet for tablet 750be79c62374baea26e714cf95fae21 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 15:01:37 UTC) from {username='kudu', principal='kudu/127.13.75.190@KRBTEST.COM'} at 127.0.0.1:52236
I20260923 15:01:37.845202 15261 tablet_replica.cc:330] T 750be79c62374baea26e714cf95fae21 P 1cebda21ec684fcabfa5beaef4edda08: stopping tablet replica
I20260923 15:01:37.845248 15261 raft_consensus.cc:2229] T 750be79c62374baea26e714cf95fae21 P 1cebda21ec684fcabfa5beaef4edda08 [term 1 LEADER]: Raft consensus shutting down.
I20260923 15:01:37.845249 15147 tablet_service.cc:1514] Processing DeleteTablet for tablet 750be79c62374baea26e714cf95fae21 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 15:01:37 UTC) from {username='kudu', principal='kudu/127.13.75.190@KRBTEST.COM'} at 127.0.0.1:51358
I20260923 15:01:37.845340 15261 raft_consensus.cc:2258] T 750be79c62374baea26e714cf95fae21 P 1cebda21ec684fcabfa5beaef4edda08 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 15:01:37.845345 15259 tablet_replica.cc:330] T 750be79c62374baea26e714cf95fae21 P 280fe8782f3a420a917580684f455c7a: stopping tablet replica
I20260923 15:01:37.845398 15259 raft_consensus.cc:2229] T 750be79c62374baea26e714cf95fae21 P 280fe8782f3a420a917580684f455c7a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 15:01:37.845441 15259 raft_consensus.cc:2258] T 750be79c62374baea26e714cf95fae21 P 280fe8782f3a420a917580684f455c7a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 15:01:37.845783 15261 ts_tablet_manager.cc:1905] T 750be79c62374baea26e714cf95fae21 P 1cebda21ec684fcabfa5beaef4edda08: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 15:01:37.845880 15259 ts_tablet_manager.cc:1905] T 750be79c62374baea26e714cf95fae21 P 280fe8782f3a420a917580684f455c7a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 15:01:37.847635 15259 ts_tablet_manager.cc:1918] T 750be79c62374baea26e714cf95fae21 P 280fe8782f3a420a917580684f455c7a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260923 15:01:37.847628 15007 tablet_service.cc:1514] Processing DeleteTablet for tablet 750be79c62374baea26e714cf95fae21 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 15:01:37 UTC) from {username='kudu', principal='kudu/127.13.75.190@KRBTEST.COM'} at 127.0.0.1:49888
I20260923 15:01:37.847698 15259 log.cc:1196] T 750be79c62374baea26e714cf95fae21 P 280fe8782f3a420a917580684f455c7a: Deleting WAL directory at /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175691491353-13614-0/minicluster-data/ts-2/wal/wals/750be79c62374baea26e714cf95fae21
I20260923 15:01:37.847914 15259 ts_tablet_manager.cc:1939] T 750be79c62374baea26e714cf95fae21 P 280fe8782f3a420a917580684f455c7a: Deleting consensus metadata
I20260923 15:01:37.848188 15260 tablet_replica.cc:330] T 750be79c62374baea26e714cf95fae21 P d046a0f9cc58427986a4fa16815e54e6: stopping tablet replica
I20260923 15:01:37.848244 15260 raft_consensus.cc:2229] T 750be79c62374baea26e714cf95fae21 P d046a0f9cc58427986a4fa16815e54e6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 15:01:37.848284 15260 raft_consensus.cc:2258] T 750be79c62374baea26e714cf95fae21 P d046a0f9cc58427986a4fa16815e54e6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 15:01:37.848523 14717 catalog_manager.cc:4841] TS 280fe8782f3a420a917580684f455c7a (127.13.75.131:34765): tablet 750be79c62374baea26e714cf95fae21 (table test-table [id=2bd7255e334d4240aa2a25663d5a094f]) successfully deleted
I20260923 15:01:37.848685 15261 ts_tablet_manager.cc:1918] T 750be79c62374baea26e714cf95fae21 P 1cebda21ec684fcabfa5beaef4edda08: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260923 15:01:37.848734 15261 log.cc:1196] T 750be79c62374baea26e714cf95fae21 P 1cebda21ec684fcabfa5beaef4edda08: Deleting WAL directory at /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175691491353-13614-0/minicluster-data/ts-0/wal/wals/750be79c62374baea26e714cf95fae21
I20260923 15:01:37.848820 15260 ts_tablet_manager.cc:1905] T 750be79c62374baea26e714cf95fae21 P d046a0f9cc58427986a4fa16815e54e6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 15:01:37.848961 15261 ts_tablet_manager.cc:1939] T 750be79c62374baea26e714cf95fae21 P 1cebda21ec684fcabfa5beaef4edda08: Deleting consensus metadata
I20260923 15:01:37.849269 14716 catalog_manager.cc:4841] TS 1cebda21ec684fcabfa5beaef4edda08 (127.13.75.129:43089): tablet 750be79c62374baea26e714cf95fae21 (table test-table [id=2bd7255e334d4240aa2a25663d5a094f]) successfully deleted
I20260923 15:01:37.850411 15260 ts_tablet_manager.cc:1918] T 750be79c62374baea26e714cf95fae21 P d046a0f9cc58427986a4fa16815e54e6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260923 15:01:37.850466 15260 log.cc:1196] T 750be79c62374baea26e714cf95fae21 P d046a0f9cc58427986a4fa16815e54e6: Deleting WAL directory at /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.SmokeTestAsAuthorizedUser.1790175691491353-13614-0/minicluster-data/ts-1/wal/wals/750be79c62374baea26e714cf95fae21
I20260923 15:01:37.850668 15260 ts_tablet_manager.cc:1939] T 750be79c62374baea26e714cf95fae21 P d046a0f9cc58427986a4fa16815e54e6: Deleting consensus metadata
I20260923 15:01:37.851171 14717 catalog_manager.cc:4841] TS d046a0f9cc58427986a4fa16815e54e6 (127.13.75.130:35027): tablet 750be79c62374baea26e714cf95fae21 (table test-table [id=2bd7255e334d4240aa2a25663d5a094f]) successfully deleted
Sep 23 15:01:37 dist-test-slave-cnb0 krb5kdc[14685](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175697, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.13.75.129@KRBTEST.COM
I20260923 15:01:37.868712 15234 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:37.855204 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52258 (local address 127.13.75.129:43089)
0923 15:01:37.855249 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0923 15:01:37.855252 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:01:37.855415 (+ 163us) server_negotiation.cc:407] Connection header received
0923 15:01:37.855485 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:01:37.855486 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:01:37.855498 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:01:37.855563 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:01:37.856250 (+ 687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:37.857000 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:37.858071 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:37.858360 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:37.858434 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:01:37.865013 (+ 6579us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:01:37.865028 (+ 15us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:01:37.865029 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:01:37.865040 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:01:37.867221 (+ 2181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:01:37.867744 (+ 523us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:01:37.867748 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:01:37.867749 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:01:37.867817 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:01:37.868051 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:01:37.868052 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:01:37.868052 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:01:37.868233 (+ 181us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:01:37.868379 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0923 15:01:37.868484 (+ 105us) server_negotiation.cc:299] Negotiation successful
0923 15:01:37.868581 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
W20260923 15:01:37.868959 14927 server_base.cc:1116] Unauthorized access attempt to method kudu.server.GenericService.SetFlag from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:52258
I20260923 15:01:37.877765 14781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:37.869969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45320 (local address 127.13.75.190:35253)
0923 15:01:37.870015 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0923 15:01:37.870017 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:01:37.870213 (+ 196us) server_negotiation.cc:407] Connection header received
0923 15:01:37.870270 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:01:37.870271 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:01:37.870284 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:01:37.870346 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:01:37.871111 (+ 765us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:37.871957 (+ 846us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:37.872935 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:37.873246 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:37.873319 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:01:37.874814 (+ 1495us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:01:37.874815 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:01:37.874816 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:01:37.874824 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:01:37.876521 (+ 1697us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:01:37.876937 (+ 416us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:01:37.876938 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:01:37.876939 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:01:37.876997 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:01:37.877214 (+ 217us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:01:37.877215 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:01:37.877215 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:01:37.877350 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:01:37.877447 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0923 15:01:37.877585 (+ 138us) server_negotiation.cc:299] Negotiation successful
0923 15:01:37.877665 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
W20260923 15:01:37.878072 14729 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:45320
I20260923 15:01:37.881302 13614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKC4soy/build/release/bin/kudu with pid 14802
I20260923 15:01:37.887696 13614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKC4soy/build/release/bin/kudu with pid 14942
I20260923 15:01:37.893584 13614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKC4soy/build/release/bin/kudu with pid 15082
I20260923 15:01:37.899267 13614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKC4soy/build/release/bin/kudu with pid 14698
2026-09-23T15:01:37Z chronyd exiting
[ OK ] SecurityITest.SmokeTestAsAuthorizedUser (2371 ms)
[ RUN ] SecurityITest.TxnSmokeWithDifferentUserTypes
Loading random data
Initializing database '/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175691491353-13614-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 15:01:37 dist-test-slave-cnb0 krb5kdc[15304](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 15:01:37 dist-test-slave-cnb0 krb5kdc[15304](info): set up 2 sockets
Sep 23 15:01:37 dist-test-slave-cnb0 krb5kdc[15304](info): commencing operation
krb5kdc: starting...
W20260923 15:01:38.944026 13614 mini_kdc.cc:121] Time spent starting KDC: real 1.024s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175691491353-13614-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175691491353-13614-0/krb5kdc/test-user.keytab.
Sep 23 15:01:39 dist-test-slave-cnb0 krb5kdc[15304](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175699, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T15:01:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T15:01:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.75.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.75.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.75.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.190.keytab.
Entry for principal kudu/127.13.75.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.190.keytab.
I20260923 15:01:39.070500 13614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKC4soy/build/release/bin/kudu
/tmp/dist-test-taskKC4soy/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175691491353-13614-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175691491353-13614-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175691491353-13614-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175691491353-13614-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.75.190:43703
--webserver_interface=127.13.75.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.190.keytab
--principal=kudu/127.13.75.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.75.148:38307
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.75.190:43703
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175691491353-13614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175691491353-13614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175691491353-13614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:01:39.154546 15317 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:01:39.154843 15317 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:01:39.154917 15317 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:01:39.156563 15317 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 15:01:39.156633 15317 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260923 15:01:39.156649 15317 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:01:39.156665 15317 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:01:39.156679 15317 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 15:01:39.156693 15317 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 15:01:39.158528 15317 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.75.148:38307
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175691491353-13614-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175691491353-13614-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.75.190:43703
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.190.keytab
--principal=kudu/127.13.75.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.75.190:43703
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175691491353-13614-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.75.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15317
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175691491353-13614-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:01:39.158780 15317 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:01:39.159003 15317 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:01:39.164227 15325 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:01:39.164372 15323 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:01:39.164227 15322 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:01:39.164688 15317 server_base.cc:1034] running on GCE node
I20260923 15:01:39.164969 15317 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:01:39.165287 15317 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:01:39.166443 15317 hybrid_clock.cc:648] HybridClock initialized: now 1790175699166416 us; error 53 us; skew 500 ppm
Sep 23 15:01:39 dist-test-slave-cnb0 krb5kdc[15304](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175699, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:01:39.169303 15317 init.cc:377] Logged in from keytab as kudu/127.13.75.190@KRBTEST.COM (short username kudu)
I20260923 15:01:39.169800 15317 webserver.cc:460] Webserver started at http://127.13.75.190:40001/ using document root <none> and password file <none>
I20260923 15:01:39.170069 15317 fs_manager.cc:359] Metadata directory not provided
I20260923 15:01:39.170145 15317 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:01:39.170300 15317 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:01:39.171527 15317 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175691491353-13614-0/minicluster-data/master-0/data/instance:
uuid: "413f7cf8ddb348c7a6a08580e4599d5b"
format_stamp: "Formatted at 2026-09-23 15:01:39 on dist-test-slave-cnb0"
server_key: "5cb9e2541e906660e350fa2d0e357fd9"
server_key_iv: "5dd345530e1f4ad9e9ac0fe179ef8545"
server_key_version: "encryptionkey@0"
I20260923 15:01:39.171933 15317 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175691491353-13614-0/minicluster-data/master-0/wal/instance:
uuid: "413f7cf8ddb348c7a6a08580e4599d5b"
format_stamp: "Formatted at 2026-09-23 15:01:39 on dist-test-slave-cnb0"
server_key: "5cb9e2541e906660e350fa2d0e357fd9"
server_key_iv: "5dd345530e1f4ad9e9ac0fe179ef8545"
server_key_version: "encryptionkey@0"
I20260923 15:01:39.173411 15317 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 15:01:39.174260 15332 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:01:39.174468 15317 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:01:39.174582 15317 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175691491353-13614-0/minicluster-data/master-0/data,/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175691491353-13614-0/minicluster-data/master-0/wal
uuid: "413f7cf8ddb348c7a6a08580e4599d5b"
format_stamp: "Formatted at 2026-09-23 15:01:39 on dist-test-slave-cnb0"
server_key: "5cb9e2541e906660e350fa2d0e357fd9"
server_key_iv: "5dd345530e1f4ad9e9ac0fe179ef8545"
server_key_version: "encryptionkey@0"
I20260923 15:01:39.174664 15317 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175691491353-13614-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175691491353-13614-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175691491353-13614-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:01:39.192842 15317 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:01:39.195564 15317 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:01:39.195740 15317 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:01:39.200804 15317 rpc_server.cc:307] RPC server started. Bound to: 127.13.75.190:43703
I20260923 15:01:39.200886 15394 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.75.190:43703 every 8 connection(s)
I20260923 15:01:39.201261 15317 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175691491353-13614-0/minicluster-data/master-0/data/info.pb
I20260923 15:01:39.201959 15395 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:01:39.204347 15395 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 413f7cf8ddb348c7a6a08580e4599d5b: Bootstrap starting.
I20260923 15:01:39.204936 15395 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 413f7cf8ddb348c7a6a08580e4599d5b: Neither blocks nor log segments found. Creating new log.
I20260923 15:01:39.205221 15395 log.cc:824] T 00000000000000000000000000000000 P 413f7cf8ddb348c7a6a08580e4599d5b: Log is configured to *not* fsync() on all Append() calls
I20260923 15:01:39.205610 13614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKC4soy/build/release/bin/kudu as pid 15317
I20260923 15:01:39.205689 13614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175691491353-13614-0/minicluster-data/master-0/wal/instance
I20260923 15:01:39.205917 13614 external_mini_cluster.cc:1428] Setting key 7693c87e34ba4c4ac97ad007241f55f3
I20260923 15:01:39.206140 15395 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 413f7cf8ddb348c7a6a08580e4599d5b: No bootstrap required, opened a new log
I20260923 15:01:39.209298 15395 raft_consensus.cc:348] T 00000000000000000000000000000000 P 413f7cf8ddb348c7a6a08580e4599d5b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "413f7cf8ddb348c7a6a08580e4599d5b" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 43703 } }
I20260923 15:01:39.209472 15395 raft_consensus.cc:374] T 00000000000000000000000000000000 P 413f7cf8ddb348c7a6a08580e4599d5b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:01:39.209527 15395 raft_consensus.cc:729] T 00000000000000000000000000000000 P 413f7cf8ddb348c7a6a08580e4599d5b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 413f7cf8ddb348c7a6a08580e4599d5b, State: Initialized, Role: FOLLOWER
I20260923 15:01:39.209669 15395 consensus_queue.cc:260] T 00000000000000000000000000000000 P 413f7cf8ddb348c7a6a08580e4599d5b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "413f7cf8ddb348c7a6a08580e4599d5b" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 43703 } }
I20260923 15:01:39.209774 15395 raft_consensus.cc:388] T 00000000000000000000000000000000 P 413f7cf8ddb348c7a6a08580e4599d5b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 15:01:39.209823 15395 raft_consensus.cc:482] T 00000000000000000000000000000000 P 413f7cf8ddb348c7a6a08580e4599d5b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 15:01:39.209875 15395 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 413f7cf8ddb348c7a6a08580e4599d5b [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:01:39.210489 15395 raft_consensus.cc:504] T 00000000000000000000000000000000 P 413f7cf8ddb348c7a6a08580e4599d5b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "413f7cf8ddb348c7a6a08580e4599d5b" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 43703 } }
I20260923 15:01:39.210628 15395 leader_election.cc:302] T 00000000000000000000000000000000 P 413f7cf8ddb348c7a6a08580e4599d5b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 413f7cf8ddb348c7a6a08580e4599d5b; no voters:
I20260923 15:01:39.210829 15395 leader_election.cc:288] T 00000000000000000000000000000000 P 413f7cf8ddb348c7a6a08580e4599d5b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 15:01:39.210858 15400 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 413f7cf8ddb348c7a6a08580e4599d5b [term 1 FOLLOWER]: Leader election won for term 1
I20260923 15:01:39.211050 15400 raft_consensus.cc:686] T 00000000000000000000000000000000 P 413f7cf8ddb348c7a6a08580e4599d5b [term 1 LEADER]: Becoming Leader. State: Replica: 413f7cf8ddb348c7a6a08580e4599d5b, State: Running, Role: LEADER
I20260923 15:01:39.211171 15395 sys_catalog.cc:564] T 00000000000000000000000000000000 P 413f7cf8ddb348c7a6a08580e4599d5b [sys.catalog]: configured and running, proceeding with master startup.
I20260923 15:01:39.211176 15400 consensus_queue.cc:237] T 00000000000000000000000000000000 P 413f7cf8ddb348c7a6a08580e4599d5b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "413f7cf8ddb348c7a6a08580e4599d5b" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 43703 } }
I20260923 15:01:39.211555 15402 sys_catalog.cc:455] T 00000000000000000000000000000000 P 413f7cf8ddb348c7a6a08580e4599d5b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 413f7cf8ddb348c7a6a08580e4599d5b. Latest consensus state: current_term: 1 leader_uuid: "413f7cf8ddb348c7a6a08580e4599d5b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "413f7cf8ddb348c7a6a08580e4599d5b" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 43703 } } }
I20260923 15:01:39.211633 15402 sys_catalog.cc:458] T 00000000000000000000000000000000 P 413f7cf8ddb348c7a6a08580e4599d5b [sys.catalog]: This master's current role is: LEADER
I20260923 15:01:39.211903 15401 sys_catalog.cc:455] T 00000000000000000000000000000000 P 413f7cf8ddb348c7a6a08580e4599d5b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "413f7cf8ddb348c7a6a08580e4599d5b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "413f7cf8ddb348c7a6a08580e4599d5b" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 43703 } } }
I20260923 15:01:39.212052 15401 sys_catalog.cc:458] T 00000000000000000000000000000000 P 413f7cf8ddb348c7a6a08580e4599d5b [sys.catalog]: This master's current role is: LEADER
I20260923 15:01:39.212523 15409 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 15:01:39.213099 15409 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Sep 23 15:01:39 dist-test-slave-cnb0 krb5kdc[15304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175699, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.75.190@KRBTEST.COM
I20260923 15:01:39.214790 15409 catalog_manager.cc:1347] Generated new cluster ID: 4ba0b820a37e4f95a95ac21dedf9ec08
I20260923 15:01:39.214843 15409 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 15:01:39.218736 15399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:39.206452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45996 (local address 127.13.75.190:43703)
0923 15:01:39.206617 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0923 15:01:39.206626 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0923 15:01:39.206658 (+ 32us) server_negotiation.cc:407] Connection header received
0923 15:01:39.207646 (+ 988us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:01:39.207653 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:01:39.207703 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:01:39.207820 (+ 117us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:01:39.208744 (+ 924us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:39.209752 (+ 1008us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:39.210579 (+ 827us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:39.211354 (+ 775us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:39.211522 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:01:39.214460 (+ 2938us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:01:39.214464 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:01:39.214464 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:01:39.214477 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:01:39.216631 (+ 2154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:01:39.217057 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:01:39.217060 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:01:39.217061 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:01:39.217170 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:01:39.217385 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:01:39.217386 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:01:39.217386 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:01:39.217701 (+ 315us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:01:39.218312 (+ 611us) server_negotiation.cc:1035] Waiting for connection context
0923 15:01:39.218381 (+ 69us) server_negotiation.cc:299] Negotiation successful
0923 15:01:39.218521 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":77,"threads_started":1}
I20260923 15:01:39.235303 15409 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 15:01:39.235939 15409 catalog_manager.cc:1504] Loading token signing keys...
I20260923 15:01:39.242877 15409 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 413f7cf8ddb348c7a6a08580e4599d5b: Generated new TSK 0
I20260923 15:01:39.243119 15409 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.75.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.75.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.75.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.129.keytab.
Entry for principal kudu/127.13.75.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.129.keytab.
I20260923 15:01:39.279419 13614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKC4soy/build/release/bin/kudu
/tmp/dist-test-taskKC4soy/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175691491353-13614-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175691491353-13614-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175691491353-13614-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175691491353-13614-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.75.129:0
--local_ip_for_outbound_sockets=127.13.75.129
--webserver_interface=127.13.75.129
--webserver_port=0
--tserver_master_addrs=127.13.75.190:43703
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.129.keytab
--principal=kudu/127.13.75.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.75.148:38307
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175691491353-13614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175691491353-13614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175691491353-13614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:01:39.373281 15421 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:01:39.373492 15421 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:01:39.373517 15421 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:01:39.374734 15421 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:01:39.374773 15421 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:01:39.374794 15421 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260923 15:01:39.374806 15421 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260923 15:01:39.374843 15421 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.75.129
I20260923 15:01:39.376343 15421 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.75.148:38307
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175691491353-13614-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175691491353-13614-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.129.keytab
--principal=kudu/127.13.75.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.75.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175691491353-13614-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.75.129
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.13.75.190:43703
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15421
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.75.129
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175691491353-13614-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:01:39.376590 15421 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:01:39.376840 15421 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:01:39.382275 15426 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:01:39.382275 15429 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:01:39.382289 15427 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:01:39.382587 15421 server_base.cc:1034] running on GCE node
I20260923 15:01:39.382773 15421 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:01:39.383033 15421 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:01:39.384220 15421 hybrid_clock.cc:648] HybridClock initialized: now 1790175699384203 us; error 34 us; skew 500 ppm
Sep 23 15:01:39 dist-test-slave-cnb0 krb5kdc[15304](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175699, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:01:39.386955 15421 init.cc:377] Logged in from keytab as kudu/127.13.75.129@KRBTEST.COM (short username kudu)
I20260923 15:01:39.387392 15421 webserver.cc:460] Webserver started at http://127.13.75.129:41831/ using document root <none> and password file <none>
I20260923 15:01:39.387641 15421 fs_manager.cc:359] Metadata directory not provided
I20260923 15:01:39.387712 15421 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:01:39.387856 15421 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:01:39.388957 15421 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175691491353-13614-0/minicluster-data/ts-0/data/instance:
uuid: "6e03dc9443b94cd09a97dc52085ffead"
format_stamp: "Formatted at 2026-09-23 15:01:39 on dist-test-slave-cnb0"
server_key: "0d0df458d37c94727a51c794d77707ec"
server_key_iv: "39e322d0fa38648ab5737e7573248577"
server_key_version: "encryptionkey@0"
I20260923 15:01:39.389314 15421 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175691491353-13614-0/minicluster-data/ts-0/wal/instance:
uuid: "6e03dc9443b94cd09a97dc52085ffead"
format_stamp: "Formatted at 2026-09-23 15:01:39 on dist-test-slave-cnb0"
server_key: "0d0df458d37c94727a51c794d77707ec"
server_key_iv: "39e322d0fa38648ab5737e7573248577"
server_key_version: "encryptionkey@0"
I20260923 15:01:39.390652 15421 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260923 15:01:39.391441 15436 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:01:39.391608 15421 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260923 15:01:39.391708 15421 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175691491353-13614-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175691491353-13614-0/minicluster-data/ts-0/wal
uuid: "6e03dc9443b94cd09a97dc52085ffead"
format_stamp: "Formatted at 2026-09-23 15:01:39 on dist-test-slave-cnb0"
server_key: "0d0df458d37c94727a51c794d77707ec"
server_key_iv: "39e322d0fa38648ab5737e7573248577"
server_key_version: "encryptionkey@0"
I20260923 15:01:39.391786 15421 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175691491353-13614-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175691491353-13614-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175691491353-13614-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:01:39.406479 15421 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:01:39.408663 15421 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:01:39.408824 15421 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:01:39.409329 15421 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 15:01:39.409389 15421 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 15:01:39.409435 15421 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 15:01:39.409482 15421 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 15:01:39.416195 15421 rpc_server.cc:307] RPC server started. Bound to: 127.13.75.129:36423
I20260923 15:01:39.416266 15551 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.75.129:36423 every 8 connection(s)
I20260923 15:01:39.416574 15421 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175691491353-13614-0/minicluster-data/ts-0/data/info.pb
Sep 23 15:01:39 dist-test-slave-cnb0 krb5kdc[15304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175699, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.129@KRBTEST.COM for kudu/127.13.75.190@KRBTEST.COM
I20260923 15:01:39.420221 15399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:39.409917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.129:36209 (local address 127.13.75.190:43703)
0923 15:01:39.409978 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0923 15:01:39.409980 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:01:39.410194 (+ 214us) server_negotiation.cc:407] Connection header received
0923 15:01:39.410774 (+ 580us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:01:39.410777 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:01:39.410791 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:01:39.410848 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:01:39.411862 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:39.412747 (+ 885us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:39.414331 (+ 1584us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:39.414698 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:39.414779 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:01:39.417404 (+ 2625us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:01:39.417405 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:01:39.417406 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:01:39.417414 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:01:39.418817 (+ 1403us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:01:39.419243 (+ 426us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:01:39.419245 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:01:39.419245 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:01:39.419302 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:01:39.419566 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:01:39.419567 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:01:39.419567 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:01:39.419708 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:01:39.419771 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0923 15:01:39.420046 (+ 275us) server_negotiation.cc:299] Negotiation successful
0923 15:01:39.420130 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260923 15:01:39.420414 15445 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:39.409885 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:43703 (local address 127.13.75.129:36209)
0923 15:01:39.410061 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0923 15:01:39.410107 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0923 15:01:39.410664 (+ 557us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:01:39.410952 (+ 288us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:01:39.410956 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:01:39.411448 (+ 492us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:01:39.411746 (+ 298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:01:39.411753 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:01:39.412849 (+ 1096us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:01:39.412850 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:01:39.414232 (+ 1382us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:01:39.414236 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:01:39.414811 (+ 575us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:01:39.414812 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:01:39.415003 (+ 191us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:01:39.415612 (+ 609us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:01:39.415632 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:01:39.417304 (+ 1672us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:01:39.418920 (+ 1616us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:01:39.418929 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:01:39.418931 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:01:39.419154 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:01:39.419372 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:01:39.419372 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:01:39.419373 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:01:39.419495 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:01:39.419762 (+ 267us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:01:39.419768 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:01:39.419911 (+ 143us) client_negotiation.cc:769] Sending connection context
0923 15:01:39.420010 (+ 99us) client_negotiation.cc:240] Negotiation successful
0923 15:01:39.420173 (+ 163us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":94,"threads_started":1}
I20260923 15:01:39.421002 15552 heartbeater.cc:347] Connected to a master server at 127.13.75.190:43703
I20260923 15:01:39.421082 15552 heartbeater.cc:464] Registering TS with master...
I20260923 15:01:39.421271 15552 heartbeater.cc:511] Master 127.13.75.190:43703 requested a full tablet report, sending...
I20260923 15:01:39.421900 15348 ts_manager.cc:194] Registered new tserver with Master: 6e03dc9443b94cd09a97dc52085ffead (127.13.75.129:36423)
I20260923 15:01:39.422809 15348 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.75.129@KRBTEST.COM'} at 127.13.75.129:36209
I20260923 15:01:39.425138 13614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKC4soy/build/release/bin/kudu as pid 15421
I20260923 15:01:39.425233 13614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175691491353-13614-0/minicluster-data/ts-0/wal/instance
I20260923 15:01:39.425422 13614 external_mini_cluster.cc:1428] Setting key 2727de72f956be58507bedbefd5d2dc6
I20260923 15:01:39.427937 15558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:39.422038 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:43703 (local address 127.13.75.129:56643)
0923 15:01:39.422184 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0923 15:01:39.422194 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 15:01:39.422256 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:01:39.422389 (+ 133us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:01:39.422390 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:01:39.422579 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:01:39.422639 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:01:39.422641 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:01:39.423371 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:01:39.423372 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:01:39.423923 (+ 551us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:01:39.423924 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:01:39.424294 (+ 370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:01:39.424295 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:01:39.424365 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:01:39.424745 (+ 380us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:01:39.424749 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:01:39.425103 (+ 354us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:01:39.426795 (+ 1692us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:01:39.426796 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:01:39.426797 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:01:39.427058 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:01:39.427280 (+ 222us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:01:39.427281 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:01:39.427281 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:01:39.427369 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:01:39.427615 (+ 246us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:01:39.427616 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:01:39.427669 (+ 53us) client_negotiation.cc:769] Sending connection context
0923 15:01:39.427749 (+ 80us) client_negotiation.cc:240] Negotiation successful
0923 15:01:39.427864 (+ 115us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":118,"thread_start_us":66,"threads_started":1}
I20260923 15:01:39.428002 15399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:39.422078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.129:56643 (local address 127.13.75.190:43703)
0923 15:01:39.422119 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0923 15:01:39.422121 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:01:39.422257 (+ 136us) server_negotiation.cc:407] Connection header received
0923 15:01:39.422314 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:01:39.422315 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:01:39.422324 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:01:39.422367 (+ 43us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:01:39.422697 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:39.423284 (+ 587us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:39.424010 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:39.424213 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:39.424293 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:01:39.425211 (+ 918us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:01:39.425213 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:01:39.425213 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:01:39.425222 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:01:39.426712 (+ 1490us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:01:39.427143 (+ 431us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:01:39.427144 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:01:39.427144 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:01:39.427201 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:01:39.427423 (+ 222us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:01:39.427423 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:01:39.427423 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:01:39.427517 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:01:39.427628 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0923 15:01:39.427792 (+ 164us) server_negotiation.cc:299] Negotiation successful
0923 15:01:39.427883 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
WARNING: no policy specified for kudu/127.13.75.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.75.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.75.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.130.keytab.
Entry for principal kudu/127.13.75.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.130.keytab.
I20260923 15:01:39.453841 13614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKC4soy/build/release/bin/kudu
/tmp/dist-test-taskKC4soy/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175691491353-13614-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175691491353-13614-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175691491353-13614-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175691491353-13614-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.75.130:0
--local_ip_for_outbound_sockets=127.13.75.130
--webserver_interface=127.13.75.130
--webserver_port=0
--tserver_master_addrs=127.13.75.190:43703
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.130.keytab
--principal=kudu/127.13.75.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.75.148:38307
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175691491353-13614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175691491353-13614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175691491353-13614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:01:39.543924 15561 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:01:39.544193 15561 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:01:39.544240 15561 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:01:39.545564 15561 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:01:39.545637 15561 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:01:39.545675 15561 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260923 15:01:39.545707 15561 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260923 15:01:39.545744 15561 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.75.130
I20260923 15:01:39.547415 15561 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.75.148:38307
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175691491353-13614-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175691491353-13614-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.130.keytab
--principal=kudu/127.13.75.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.75.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175691491353-13614-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.75.130
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.13.75.190:43703
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15561
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.75.130
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175691491353-13614-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:01:39.547704 15561 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:01:39.547959 15561 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260923 15:01:39.553404 15561 server_base.cc:1034] running on GCE node
W20260923 15:01:39.553385 15567 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:01:39.553356 15566 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:01:39.553359 15569 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:01:39.553841 15561 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:01:39.554067 15561 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:01:39.555328 15561 hybrid_clock.cc:648] HybridClock initialized: now 1790175699555308 us; error 34 us; skew 500 ppm
Sep 23 15:01:39 dist-test-slave-cnb0 krb5kdc[15304](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175699, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:01:39.558239 15561 init.cc:377] Logged in from keytab as kudu/127.13.75.130@KRBTEST.COM (short username kudu)
I20260923 15:01:39.558815 15561 webserver.cc:460] Webserver started at http://127.13.75.130:35059/ using document root <none> and password file <none>
I20260923 15:01:39.559160 15561 fs_manager.cc:359] Metadata directory not provided
I20260923 15:01:39.559239 15561 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:01:39.559412 15561 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:01:39.560509 15561 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175691491353-13614-0/minicluster-data/ts-1/data/instance:
uuid: "3e1f390e2f6b43ef846a9189c50427e8"
format_stamp: "Formatted at 2026-09-23 15:01:39 on dist-test-slave-cnb0"
server_key: "51dc16a2f5d1c77213b008791e546ab2"
server_key_iv: "92c2d0d1ed474a7435d4726cd8d69bff"
server_key_version: "encryptionkey@0"
I20260923 15:01:39.560900 15561 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175691491353-13614-0/minicluster-data/ts-1/wal/instance:
uuid: "3e1f390e2f6b43ef846a9189c50427e8"
format_stamp: "Formatted at 2026-09-23 15:01:39 on dist-test-slave-cnb0"
server_key: "51dc16a2f5d1c77213b008791e546ab2"
server_key_iv: "92c2d0d1ed474a7435d4726cd8d69bff"
server_key_version: "encryptionkey@0"
I20260923 15:01:39.562285 15561 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260923 15:01:39.563189 15576 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:01:39.563375 15561 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260923 15:01:39.563477 15561 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175691491353-13614-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175691491353-13614-0/minicluster-data/ts-1/wal
uuid: "3e1f390e2f6b43ef846a9189c50427e8"
format_stamp: "Formatted at 2026-09-23 15:01:39 on dist-test-slave-cnb0"
server_key: "51dc16a2f5d1c77213b008791e546ab2"
server_key_iv: "92c2d0d1ed474a7435d4726cd8d69bff"
server_key_version: "encryptionkey@0"
I20260923 15:01:39.563550 15561 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175691491353-13614-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175691491353-13614-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175691491353-13614-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:01:39.585379 15561 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:01:39.587898 15561 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:01:39.588063 15561 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:01:39.588603 15561 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 15:01:39.588665 15561 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 15:01:39.588697 15561 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 15:01:39.588726 15561 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 15:01:39.595311 15561 rpc_server.cc:307] RPC server started. Bound to: 127.13.75.130:37633
I20260923 15:01:39.595394 15691 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.75.130:37633 every 8 connection(s)
Sep 23 15:01:39 dist-test-slave-cnb0 krb5kdc[15304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175699, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.130@KRBTEST.COM for kudu/127.13.75.190@KRBTEST.COM
I20260923 15:01:39.595759 15561 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175691491353-13614-0/minicluster-data/ts-1/data/info.pb
I20260923 15:01:39.598929 15399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:39.589141 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.130:36605 (local address 127.13.75.190:43703)
0923 15:01:39.589212 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0923 15:01:39.589215 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:01:39.589422 (+ 207us) server_negotiation.cc:407] Connection header received
0923 15:01:39.589965 (+ 543us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:01:39.589966 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:01:39.589978 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:01:39.590018 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:01:39.590917 (+ 899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:39.591457 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:39.593071 (+ 1614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:39.593391 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:39.593484 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:01:39.596263 (+ 2779us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:01:39.596265 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:01:39.596265 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:01:39.596274 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:01:39.597449 (+ 1175us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:01:39.597928 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:01:39.597929 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:01:39.597930 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:01:39.597990 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:01:39.598240 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:01:39.598241 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:01:39.598241 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:01:39.598368 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:01:39.598445 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0923 15:01:39.598722 (+ 277us) server_negotiation.cc:299] Negotiation successful
0923 15:01:39.598810 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260923 15:01:39.599126 15585 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:39.589093 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:43703 (local address 127.13.75.130:36605)
0923 15:01:39.589290 (+ 197us) negotiation.cc:107] Waiting for socket to connect
0923 15:01:39.589331 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0923 15:01:39.589859 (+ 528us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:01:39.590109 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:01:39.590112 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:01:39.590500 (+ 388us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:01:39.590802 (+ 302us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:01:39.590810 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:01:39.591572 (+ 762us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:01:39.591573 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:01:39.592974 (+ 1401us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:01:39.592977 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:01:39.593495 (+ 518us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:01:39.593496 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:01:39.593644 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:01:39.594264 (+ 620us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:01:39.594276 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:01:39.596169 (+ 1893us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:01:39.597613 (+ 1444us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:01:39.597622 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:01:39.597625 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:01:39.597852 (+ 227us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:01:39.598061 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:01:39.598062 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:01:39.598063 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:01:39.598175 (+ 112us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:01:39.598434 (+ 259us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:01:39.598440 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:01:39.598586 (+ 146us) client_negotiation.cc:769] Sending connection context
0923 15:01:39.598687 (+ 101us) client_negotiation.cc:240] Negotiation successful
0923 15:01:39.598851 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":120,"threads_started":1}
I20260923 15:01:39.599538 13614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKC4soy/build/release/bin/kudu as pid 15561
I20260923 15:01:39.599637 13614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175691491353-13614-0/minicluster-data/ts-1/wal/instance
I20260923 15:01:39.599689 15692 heartbeater.cc:347] Connected to a master server at 127.13.75.190:43703
I20260923 15:01:39.599766 15692 heartbeater.cc:464] Registering TS with master...
I20260923 15:01:39.599845 13614 external_mini_cluster.cc:1428] Setting key 7bf63c88dffbed58399a2253347e4098
I20260923 15:01:39.599964 15692 heartbeater.cc:511] Master 127.13.75.190:43703 requested a full tablet report, sending...
I20260923 15:01:39.600332 15348 ts_manager.cc:194] Registered new tserver with Master: 3e1f390e2f6b43ef846a9189c50427e8 (127.13.75.130:37633)
I20260923 15:01:39.600840 15348 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.75.130@KRBTEST.COM'} at 127.13.75.130:36605
I20260923 15:01:39.607682 15698 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:39.600739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:43703 (local address 127.13.75.130:34775)
0923 15:01:39.600878 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0923 15:01:39.600890 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 15:01:39.600944 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:01:39.601176 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:01:39.601177 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:01:39.601458 (+ 281us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:01:39.601532 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:01:39.601535 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:01:39.602191 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:01:39.602192 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:01:39.602848 (+ 656us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:01:39.602851 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:01:39.603444 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:01:39.603444 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:01:39.603508 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:01:39.603993 (+ 485us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:01:39.603997 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:01:39.604280 (+ 283us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:01:39.606362 (+ 2082us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:01:39.606363 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:01:39.606364 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:01:39.606597 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:01:39.606810 (+ 213us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:01:39.606811 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:01:39.606811 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:01:39.606869 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:01:39.607327 (+ 458us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:01:39.607328 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:01:39.607376 (+ 48us) client_negotiation.cc:769] Sending connection context
0923 15:01:39.607505 (+ 129us) client_negotiation.cc:240] Negotiation successful
0923 15:01:39.607603 (+ 98us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":80,"threads_started":1}
I20260923 15:01:39.607749 15399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:39.600822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.130:34775 (local address 127.13.75.190:43703)
0923 15:01:39.601050 (+ 228us) server_negotiation.cc:206] Beginning negotiation
0923 15:01:39.601052 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:01:39.601059 (+ 7us) server_negotiation.cc:407] Connection header received
0923 15:01:39.601077 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:01:39.601077 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:01:39.601086 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:01:39.601130 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:01:39.601624 (+ 494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:39.602096 (+ 472us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:39.602975 (+ 879us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:39.603319 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:39.603395 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:01:39.604369 (+ 974us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:01:39.604371 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:01:39.604371 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:01:39.604378 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:01:39.606254 (+ 1876us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:01:39.606675 (+ 421us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:01:39.606676 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:01:39.606678 (+ 2us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:01:39.606740 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:01:39.606986 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:01:39.606987 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:01:39.606988 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:01:39.607218 (+ 230us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:01:39.607297 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0923 15:01:39.607538 (+ 241us) server_negotiation.cc:299] Negotiation successful
0923 15:01:39.607626 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":209}
WARNING: no policy specified for kudu/127.13.75.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.75.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.75.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.131.keytab.
Entry for principal kudu/127.13.75.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.131.keytab.
I20260923 15:01:39.629571 13614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKC4soy/build/release/bin/kudu
/tmp/dist-test-taskKC4soy/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175691491353-13614-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175691491353-13614-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175691491353-13614-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175691491353-13614-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.75.131:0
--local_ip_for_outbound_sockets=127.13.75.131
--webserver_interface=127.13.75.131
--webserver_port=0
--tserver_master_addrs=127.13.75.190:43703
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.131.keytab
--principal=kudu/127.13.75.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.75.148:38307
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true
--txn_keepalive_interval_ms=500 with env {KRB5CCNAME=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175691491353-13614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175691491353-13614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175691491353-13614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:01:39.722061 15701 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:01:39.722306 15701 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:01:39.722349 15701 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:01:39.723611 15701 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:01:39.723675 15701 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:01:39.723711 15701 flags.cc:425] Enabled experimental flag: --txn_keepalive_interval_ms=500
W20260923 15:01:39.723752 15701 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260923 15:01:39.723793 15701 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.75.131
I20260923 15:01:39.725289 15701 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.75.148:38307
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175691491353-13614-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175691491353-13614-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.131.keytab
--principal=kudu/127.13.75.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.75.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175691491353-13614-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.75.131
--webserver_port=0
--txn_keepalive_interval_ms=500
--enable_txn_system_client_init=true
--tserver_master_addrs=127.13.75.190:43703
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.15701
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.75.131
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175691491353-13614-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:01:39.725570 15701 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:01:39.725795 15701 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:01:39.731333 15709 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:01:39.731459 15701 server_base.cc:1034] running on GCE node
W20260923 15:01:39.731352 15707 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:01:39.731356 15706 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:01:39.731820 15701 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:01:39.732097 15701 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:01:39.733237 15701 hybrid_clock.cc:648] HybridClock initialized: now 1790175699733221 us; error 31 us; skew 500 ppm
Sep 23 15:01:39 dist-test-slave-cnb0 krb5kdc[15304](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175699, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:01:39.735780 15701 init.cc:377] Logged in from keytab as kudu/127.13.75.131@KRBTEST.COM (short username kudu)
I20260923 15:01:39.736253 15701 webserver.cc:460] Webserver started at http://127.13.75.131:35493/ using document root <none> and password file <none>
I20260923 15:01:39.736521 15701 fs_manager.cc:359] Metadata directory not provided
I20260923 15:01:39.736568 15701 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:01:39.736666 15701 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:01:39.737763 15701 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175691491353-13614-0/minicluster-data/ts-2/data/instance:
uuid: "2cd60cb8df9646af83deb3452da89de2"
format_stamp: "Formatted at 2026-09-23 15:01:39 on dist-test-slave-cnb0"
server_key: "c93f88738c11ce090eb74eccd68a2c1b"
server_key_iv: "85643fc0c22c7d5123f43a0ec072919b"
server_key_version: "encryptionkey@0"
I20260923 15:01:39.738109 15701 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175691491353-13614-0/minicluster-data/ts-2/wal/instance:
uuid: "2cd60cb8df9646af83deb3452da89de2"
format_stamp: "Formatted at 2026-09-23 15:01:39 on dist-test-slave-cnb0"
server_key: "c93f88738c11ce090eb74eccd68a2c1b"
server_key_iv: "85643fc0c22c7d5123f43a0ec072919b"
server_key_version: "encryptionkey@0"
I20260923 15:01:39.739629 15701 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 15:01:39.740396 15716 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:01:39.740612 15701 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:01:39.740694 15701 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175691491353-13614-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175691491353-13614-0/minicluster-data/ts-2/wal
uuid: "2cd60cb8df9646af83deb3452da89de2"
format_stamp: "Formatted at 2026-09-23 15:01:39 on dist-test-slave-cnb0"
server_key: "c93f88738c11ce090eb74eccd68a2c1b"
server_key_iv: "85643fc0c22c7d5123f43a0ec072919b"
server_key_version: "encryptionkey@0"
I20260923 15:01:39.740753 15701 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175691491353-13614-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175691491353-13614-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175691491353-13614-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:01:39.760628 15701 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:01:39.762809 15701 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:01:39.763018 15701 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:01:39.763593 15701 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 15:01:39.763631 15701 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 15:01:39.763654 15701 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 15:01:39.763691 15701 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 15:01:39.770641 15701 rpc_server.cc:307] RPC server started. Bound to: 127.13.75.131:43295
Sep 23 15:01:39 dist-test-slave-cnb0 krb5kdc[15304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175699, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.131@KRBTEST.COM for kudu/127.13.75.190@KRBTEST.COM
I20260923 15:01:39.770730 15831 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.75.131:43295 every 8 connection(s)
I20260923 15:01:39.771103 15701 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175691491353-13614-0/minicluster-data/ts-2/data/info.pb
I20260923 15:01:39.773943 15399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:39.764180 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.131:53387 (local address 127.13.75.190:43703)
0923 15:01:39.764251 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0923 15:01:39.764253 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:01:39.764513 (+ 260us) server_negotiation.cc:407] Connection header received
0923 15:01:39.765131 (+ 618us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:01:39.765132 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:01:39.765147 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:01:39.765231 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:01:39.766147 (+ 916us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:39.767029 (+ 882us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:39.768205 (+ 1176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:39.768431 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:39.768512 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:01:39.771308 (+ 2796us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:01:39.771310 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:01:39.771310 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:01:39.771318 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:01:39.772593 (+ 1275us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:01:39.772989 (+ 396us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:01:39.772990 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:01:39.772990 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:01:39.773048 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:01:39.773302 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:01:39.773303 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:01:39.773303 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:01:39.773422 (+ 119us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:01:39.773507 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0923 15:01:39.773767 (+ 260us) server_negotiation.cc:299] Negotiation successful
0923 15:01:39.773850 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260923 15:01:39.774204 15725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:39.764157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:43703 (local address 127.13.75.131:53387)
0923 15:01:39.764405 (+ 248us) negotiation.cc:107] Waiting for socket to connect
0923 15:01:39.764434 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0923 15:01:39.764999 (+ 565us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:01:39.765321 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:01:39.765326 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:01:39.765718 (+ 392us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:01:39.766027 (+ 309us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:01:39.766035 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:01:39.767125 (+ 1090us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:01:39.767126 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:01:39.768102 (+ 976us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:01:39.768105 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:01:39.768532 (+ 427us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:01:39.768533 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:01:39.768684 (+ 151us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:01:39.769284 (+ 600us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:01:39.769296 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:01:39.771192 (+ 1896us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:01:39.772675 (+ 1483us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:01:39.772681 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:01:39.772684 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:01:39.772905 (+ 221us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:01:39.773123 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:01:39.773124 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:01:39.773124 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:01:39.773230 (+ 106us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:01:39.773498 (+ 268us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:01:39.773504 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:01:39.773641 (+ 137us) client_negotiation.cc:769] Sending connection context
0923 15:01:39.773746 (+ 105us) client_negotiation.cc:240] Negotiation successful
0923 15:01:39.773906 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":218,"thread_start_us":98,"threads_started":1}
I20260923 15:01:39.774700 15832 heartbeater.cc:347] Connected to a master server at 127.13.75.190:43703
I20260923 15:01:39.774784 15832 heartbeater.cc:464] Registering TS with master...
I20260923 15:01:39.775012 15832 heartbeater.cc:511] Master 127.13.75.190:43703 requested a full tablet report, sending...
I20260923 15:01:39.775171 13614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKC4soy/build/release/bin/kudu as pid 15701
I20260923 15:01:39.775252 13614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175691491353-13614-0/minicluster-data/ts-2/wal/instance
I20260923 15:01:39.775429 13614 external_mini_cluster.cc:1428] Setting key e315a259a63be423249d64e6fca00631
I20260923 15:01:39.775806 15349 ts_manager.cc:194] Registered new tserver with Master: 2cd60cb8df9646af83deb3452da89de2 (127.13.75.131:43295)
I20260923 15:01:39.776304 15349 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.75.131@KRBTEST.COM'} at 127.13.75.131:53387
I20260923 15:01:39.776340 13614 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260923 15:01:39.782709 15399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:39.775779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.131:59435 (local address 127.13.75.190:43703)
0923 15:01:39.775814 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0923 15:01:39.775815 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 15:01:39.776593 (+ 778us) server_negotiation.cc:407] Connection header received
0923 15:01:39.776640 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:01:39.776641 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:01:39.776651 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:01:39.776691 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:01:39.777076 (+ 385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:39.777583 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:39.778696 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:39.778989 (+ 293us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:39.779036 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:01:39.780060 (+ 1024us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:01:39.780061 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:01:39.780062 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:01:39.780069 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:01:39.781471 (+ 1402us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:01:39.781840 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:01:39.781842 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:01:39.781842 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:01:39.781897 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:01:39.782120 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:01:39.782121 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:01:39.782122 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:01:39.782277 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:01:39.782339 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0923 15:01:39.782559 (+ 220us) server_negotiation.cc:299] Negotiation successful
0923 15:01:39.782634 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":15}
I20260923 15:01:39.782755 15838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:39.775742 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:43703 (local address 127.13.75.131:59435)
0923 15:01:39.776518 (+ 776us) negotiation.cc:107] Waiting for socket to connect
0923 15:01:39.776530 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 15:01:39.776591 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:01:39.776715 (+ 124us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:01:39.776716 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:01:39.776927 (+ 211us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:01:39.776993 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:01:39.776995 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:01:39.777666 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:01:39.777667 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:01:39.778558 (+ 891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:01:39.778560 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:01:39.779115 (+ 555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:01:39.779116 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:01:39.779175 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:01:39.779640 (+ 465us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:01:39.779646 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:01:39.779943 (+ 297us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:01:39.781567 (+ 1624us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:01:39.781569 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:01:39.781569 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:01:39.781749 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:01:39.781965 (+ 216us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:01:39.781966 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:01:39.781967 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:01:39.782030 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:01:39.782345 (+ 315us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:01:39.782346 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:01:39.782402 (+ 56us) client_negotiation.cc:769] Sending connection context
0923 15:01:39.782479 (+ 77us) client_negotiation.cc:240] Negotiation successful
0923 15:01:39.782609 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":750,"thread_start_us":69,"threads_started":1}
Sep 23 15:01:39 dist-test-slave-cnb0 krb5kdc[15304](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175699, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 23 15:01:39 dist-test-slave-cnb0 krb5kdc[15304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175699, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.75.190@KRBTEST.COM
I20260923 15:01:39.800407 15399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:39.793227 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45998 (local address 127.13.75.190:43703)
0923 15:01:39.793271 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0923 15:01:39.793273 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:01:39.793282 (+ 9us) server_negotiation.cc:407] Connection header received
0923 15:01:39.793309 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:01:39.793310 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:01:39.793324 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:01:39.793385 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:01:39.794040 (+ 655us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:39.794553 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:39.795463 (+ 910us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:39.795652 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:39.795732 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:01:39.797938 (+ 2206us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:01:39.797939 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:01:39.797940 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:01:39.797948 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:01:39.799227 (+ 1279us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:01:39.799602 (+ 375us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:01:39.799603 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:01:39.799604 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:01:39.799677 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:01:39.799878 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:01:39.799879 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:01:39.799879 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:01:39.799997 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:01:39.800102 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0923 15:01:39.800214 (+ 112us) server_negotiation.cc:299] Negotiation successful
0923 15:01:39.800299 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260923 15:01:39.801875 15349 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45998:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 15:01:39.803577 15349 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 15:01:39.810277 15851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:39.806134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40180 (local address 127.13.75.129:36423)
0923 15:01:39.806644 (+ 510us) server_negotiation.cc:206] Beginning negotiation
0923 15:01:39.806648 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 15:01:39.806659 (+ 11us) server_negotiation.cc:407] Connection header received
0923 15:01:39.806699 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:01:39.806701 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:01:39.806715 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:01:39.807017 (+ 302us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:01:39.807076 (+ 59us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:39.807747 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:39.809096 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:39.809822 (+ 726us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:39.809897 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:01:39.809956 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0923 15:01:39.810115 (+ 159us) server_negotiation.cc:299] Negotiation successful
0923 15:01:39.810167 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":475,"thread_start_us":74,"threads_started":1}
I20260923 15:01:39.810247 15850 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:39.806005 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.131:43295 (local address 127.0.0.1:55714)
0923 15:01:39.806456 (+ 451us) negotiation.cc:107] Waiting for socket to connect
0923 15:01:39.806468 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 15:01:39.806536 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:01:39.806804 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:01:39.806807 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:01:39.806810 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:01:39.806906 (+ 96us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:01:39.806909 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:01:39.808235 (+ 1326us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:01:39.808236 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:01:39.809285 (+ 1049us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:01:39.809287 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:01:39.810027 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:01:39.810028 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:01:39.810088 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:01:39.810092 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 15:01:39.810136 (+ 44us) client_negotiation.cc:240] Negotiation successful
0923 15:01:39.810166 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":423,"mutex_wait_us":40,"thread_start_us":100,"threads_started":1}
I20260923 15:01:39.810247 15849 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:39.805991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.129:36423 (local address 127.0.0.1:40180)
0923 15:01:39.806164 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0923 15:01:39.806206 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0923 15:01:39.806268 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:01:39.806815 (+ 547us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:01:39.806815 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:01:39.806818 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:01:39.806937 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:01:39.806939 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:01:39.807830 (+ 891us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:01:39.807831 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:01:39.809010 (+ 1179us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:01:39.809013 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:01:39.809911 (+ 898us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:01:39.809912 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:01:39.809989 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:01:39.810003 (+ 14us) client_negotiation.cc:769] Sending connection context
0923 15:01:39.810086 (+ 83us) client_negotiation.cc:240] Negotiation successful
0923 15:01:39.810149 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":128,"threads_started":1}
I20260923 15:01:39.810493 15853 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:39.806263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.130:37633 (local address 127.0.0.1:51474)
0923 15:01:39.806481 (+ 218us) negotiation.cc:107] Waiting for socket to connect
0923 15:01:39.806491 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 15:01:39.806536 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:01:39.807046 (+ 510us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:01:39.807047 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:01:39.807049 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:01:39.807093 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:01:39.807094 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:01:39.808289 (+ 1195us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:01:39.808290 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:01:39.809295 (+ 1005us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:01:39.809297 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:01:39.810263 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:01:39.810264 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:01:39.810329 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:01:39.810333 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 15:01:39.810387 (+ 54us) client_negotiation.cc:240] Negotiation successful
0923 15:01:39.810427 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":196,"mutex_wait_us":3,"thread_start_us":85,"threads_started":1}
I20260923 15:01:39.810602 15852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:39.806251 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55714 (local address 127.13.75.131:43295)
0923 15:01:39.806427 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0923 15:01:39.806432 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 15:01:39.806537 (+ 105us) server_negotiation.cc:407] Connection header received
0923 15:01:39.806616 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:01:39.806619 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:01:39.806637 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:01:39.806710 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:01:39.807007 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:39.808145 (+ 1138us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:39.809362 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:39.809845 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:39.809934 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:01:39.809994 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0923 15:01:39.810402 (+ 408us) server_negotiation.cc:299] Negotiation successful
0923 15:01:39.810457 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":104,"threads_started":1}
I20260923 15:01:39.810702 15854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:39.806489 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51474 (local address 127.13.75.130:37633)
0923 15:01:39.806803 (+ 314us) server_negotiation.cc:206] Beginning negotiation
0923 15:01:39.806808 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 15:01:39.806817 (+ 9us) server_negotiation.cc:407] Connection header received
0923 15:01:39.806854 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:01:39.806856 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:01:39.806872 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:01:39.806971 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:01:39.807169 (+ 198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:39.808220 (+ 1051us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:39.809388 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:39.810112 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:39.810177 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:01:39.810228 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0923 15:01:39.810570 (+ 342us) server_negotiation.cc:299] Negotiation successful
0923 15:01:39.810609 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":280,"thread_start_us":137,"threads_started":1}
I20260923 15:01:39.811429 15766 tablet_service.cc:1467] Processing CreateTablet for tablet b7eb5458d9f3484d9e89d7cfd71fe5f3 (DEFAULT_TABLE table=test-table [id=709481ca616f40c1b2748e27231c4598]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 15:01:39.811414 15471 tablet_service.cc:1467] Processing CreateTablet for tablet b7eb5458d9f3484d9e89d7cfd71fe5f3 (DEFAULT_TABLE table=test-table [id=709481ca616f40c1b2748e27231c4598]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 15:01:39.811415 15626 tablet_service.cc:1467] Processing CreateTablet for tablet b7eb5458d9f3484d9e89d7cfd71fe5f3 (DEFAULT_TABLE table=test-table [id=709481ca616f40c1b2748e27231c4598]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 15:01:39.811748 15766 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b7eb5458d9f3484d9e89d7cfd71fe5f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:01:39.811766 15471 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b7eb5458d9f3484d9e89d7cfd71fe5f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:01:39.811769 15626 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b7eb5458d9f3484d9e89d7cfd71fe5f3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:01:39.814312 15855 tablet_bootstrap.cc:492] T b7eb5458d9f3484d9e89d7cfd71fe5f3 P 2cd60cb8df9646af83deb3452da89de2: Bootstrap starting.
I20260923 15:01:39.815387 15855 tablet_bootstrap.cc:654] T b7eb5458d9f3484d9e89d7cfd71fe5f3 P 2cd60cb8df9646af83deb3452da89de2: Neither blocks nor log segments found. Creating new log.
I20260923 15:01:39.815614 15856 tablet_bootstrap.cc:492] T b7eb5458d9f3484d9e89d7cfd71fe5f3 P 3e1f390e2f6b43ef846a9189c50427e8: Bootstrap starting.
I20260923 15:01:39.815876 15855 log.cc:824] T b7eb5458d9f3484d9e89d7cfd71fe5f3 P 2cd60cb8df9646af83deb3452da89de2: Log is configured to *not* fsync() on all Append() calls
I20260923 15:01:39.816257 15857 tablet_bootstrap.cc:492] T b7eb5458d9f3484d9e89d7cfd71fe5f3 P 6e03dc9443b94cd09a97dc52085ffead: Bootstrap starting.
I20260923 15:01:39.816294 15856 tablet_bootstrap.cc:654] T b7eb5458d9f3484d9e89d7cfd71fe5f3 P 3e1f390e2f6b43ef846a9189c50427e8: Neither blocks nor log segments found. Creating new log.
I20260923 15:01:39.816622 15856 log.cc:824] T b7eb5458d9f3484d9e89d7cfd71fe5f3 P 3e1f390e2f6b43ef846a9189c50427e8: Log is configured to *not* fsync() on all Append() calls
I20260923 15:01:39.816664 15855 tablet_bootstrap.cc:492] T b7eb5458d9f3484d9e89d7cfd71fe5f3 P 2cd60cb8df9646af83deb3452da89de2: No bootstrap required, opened a new log
I20260923 15:01:39.816742 15855 ts_tablet_manager.cc:1397] T b7eb5458d9f3484d9e89d7cfd71fe5f3 P 2cd60cb8df9646af83deb3452da89de2: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260923 15:01:39.816953 15857 tablet_bootstrap.cc:654] T b7eb5458d9f3484d9e89d7cfd71fe5f3 P 6e03dc9443b94cd09a97dc52085ffead: Neither blocks nor log segments found. Creating new log.
I20260923 15:01:39.817291 15857 log.cc:824] T b7eb5458d9f3484d9e89d7cfd71fe5f3 P 6e03dc9443b94cd09a97dc52085ffead: Log is configured to *not* fsync() on all Append() calls
I20260923 15:01:39.817421 15856 tablet_bootstrap.cc:492] T b7eb5458d9f3484d9e89d7cfd71fe5f3 P 3e1f390e2f6b43ef846a9189c50427e8: No bootstrap required, opened a new log
I20260923 15:01:39.817499 15856 ts_tablet_manager.cc:1397] T b7eb5458d9f3484d9e89d7cfd71fe5f3 P 3e1f390e2f6b43ef846a9189c50427e8: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 15:01:39.818117 15857 tablet_bootstrap.cc:492] T b7eb5458d9f3484d9e89d7cfd71fe5f3 P 6e03dc9443b94cd09a97dc52085ffead: No bootstrap required, opened a new log
I20260923 15:01:39.818189 15857 ts_tablet_manager.cc:1397] T b7eb5458d9f3484d9e89d7cfd71fe5f3 P 6e03dc9443b94cd09a97dc52085ffead: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 15:01:39.818703 15855 raft_consensus.cc:348] T b7eb5458d9f3484d9e89d7cfd71fe5f3 P 2cd60cb8df9646af83deb3452da89de2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e03dc9443b94cd09a97dc52085ffead" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 36423 } } peers { permanent_uuid: "3e1f390e2f6b43ef846a9189c50427e8" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 37633 } } peers { permanent_uuid: "2cd60cb8df9646af83deb3452da89de2" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 43295 } }
I20260923 15:01:39.818923 15855 raft_consensus.cc:374] T b7eb5458d9f3484d9e89d7cfd71fe5f3 P 2cd60cb8df9646af83deb3452da89de2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:01:39.818974 15855 raft_consensus.cc:729] T b7eb5458d9f3484d9e89d7cfd71fe5f3 P 2cd60cb8df9646af83deb3452da89de2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2cd60cb8df9646af83deb3452da89de2, State: Initialized, Role: FOLLOWER
I20260923 15:01:39.819103 15855 consensus_queue.cc:260] T b7eb5458d9f3484d9e89d7cfd71fe5f3 P 2cd60cb8df9646af83deb3452da89de2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e03dc9443b94cd09a97dc52085ffead" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 36423 } } peers { permanent_uuid: "3e1f390e2f6b43ef846a9189c50427e8" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 37633 } } peers { permanent_uuid: "2cd60cb8df9646af83deb3452da89de2" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 43295 } }
I20260923 15:01:39.819408 15855 ts_tablet_manager.cc:1428] T b7eb5458d9f3484d9e89d7cfd71fe5f3 P 2cd60cb8df9646af83deb3452da89de2: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260923 15:01:39.819465 15832 heartbeater.cc:503] Master 127.13.75.190:43703 was elected leader, sending a full tablet report...
I20260923 15:01:39.819972 15856 raft_consensus.cc:348] T b7eb5458d9f3484d9e89d7cfd71fe5f3 P 3e1f390e2f6b43ef846a9189c50427e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e03dc9443b94cd09a97dc52085ffead" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 36423 } } peers { permanent_uuid: "3e1f390e2f6b43ef846a9189c50427e8" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 37633 } } peers { permanent_uuid: "2cd60cb8df9646af83deb3452da89de2" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 43295 } }
I20260923 15:01:39.820147 15856 raft_consensus.cc:374] T b7eb5458d9f3484d9e89d7cfd71fe5f3 P 3e1f390e2f6b43ef846a9189c50427e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:01:39.820186 15856 raft_consensus.cc:729] T b7eb5458d9f3484d9e89d7cfd71fe5f3 P 3e1f390e2f6b43ef846a9189c50427e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e1f390e2f6b43ef846a9189c50427e8, State: Initialized, Role: FOLLOWER
I20260923 15:01:39.820344 15856 consensus_queue.cc:260] T b7eb5458d9f3484d9e89d7cfd71fe5f3 P 3e1f390e2f6b43ef846a9189c50427e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e03dc9443b94cd09a97dc52085ffead" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 36423 } } peers { permanent_uuid: "3e1f390e2f6b43ef846a9189c50427e8" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 37633 } } peers { permanent_uuid: "2cd60cb8df9646af83deb3452da89de2" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 43295 } }
I20260923 15:01:39.820307 15857 raft_consensus.cc:348] T b7eb5458d9f3484d9e89d7cfd71fe5f3 P 6e03dc9443b94cd09a97dc52085ffead [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e03dc9443b94cd09a97dc52085ffead" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 36423 } } peers { permanent_uuid: "3e1f390e2f6b43ef846a9189c50427e8" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 37633 } } peers { permanent_uuid: "2cd60cb8df9646af83deb3452da89de2" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 43295 } }
I20260923 15:01:39.820487 15857 raft_consensus.cc:374] T b7eb5458d9f3484d9e89d7cfd71fe5f3 P 6e03dc9443b94cd09a97dc52085ffead [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:01:39.820536 15857 raft_consensus.cc:729] T b7eb5458d9f3484d9e89d7cfd71fe5f3 P 6e03dc9443b94cd09a97dc52085ffead [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e03dc9443b94cd09a97dc52085ffead, State: Initialized, Role: FOLLOWER
I20260923 15:01:39.820647 15856 ts_tablet_manager.cc:1428] T b7eb5458d9f3484d9e89d7cfd71fe5f3 P 3e1f390e2f6b43ef846a9189c50427e8: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260923 15:01:39.820690 15692 heartbeater.cc:503] Master 127.13.75.190:43703 was elected leader, sending a full tablet report...
I20260923 15:01:39.820729 15857 consensus_queue.cc:260] T b7eb5458d9f3484d9e89d7cfd71fe5f3 P 6e03dc9443b94cd09a97dc52085ffead [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e03dc9443b94cd09a97dc52085ffead" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 36423 } } peers { permanent_uuid: "3e1f390e2f6b43ef846a9189c50427e8" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 37633 } } peers { permanent_uuid: "2cd60cb8df9646af83deb3452da89de2" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 43295 } }
I20260923 15:01:39.821002 15857 ts_tablet_manager.cc:1428] T b7eb5458d9f3484d9e89d7cfd71fe5f3 P 6e03dc9443b94cd09a97dc52085ffead: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260923 15:01:39.821177 15552 heartbeater.cc:503] Master 127.13.75.190:43703 was elected leader, sending a full tablet report...
W20260923 15:01:39.846485 15693 tablet.cc:2369] T b7eb5458d9f3484d9e89d7cfd71fe5f3 P 3e1f390e2f6b43ef846a9189c50427e8: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260923 15:01:39.917379 15553 tablet.cc:2369] T b7eb5458d9f3484d9e89d7cfd71fe5f3 P 6e03dc9443b94cd09a97dc52085ffead: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 15:01:39.925741 15861 raft_consensus.cc:482] T b7eb5458d9f3484d9e89d7cfd71fe5f3 P 2cd60cb8df9646af83deb3452da89de2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 15:01:39.925858 15861 raft_consensus.cc:504] T b7eb5458d9f3484d9e89d7cfd71fe5f3 P 2cd60cb8df9646af83deb3452da89de2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e03dc9443b94cd09a97dc52085ffead" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 36423 } } peers { permanent_uuid: "3e1f390e2f6b43ef846a9189c50427e8" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 37633 } } peers { permanent_uuid: "2cd60cb8df9646af83deb3452da89de2" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 43295 } }
I20260923 15:01:39.926158 15861 leader_election.cc:288] T b7eb5458d9f3484d9e89d7cfd71fe5f3 P 2cd60cb8df9646af83deb3452da89de2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e03dc9443b94cd09a97dc52085ffead (127.13.75.129:36423), 3e1f390e2f6b43ef846a9189c50427e8 (127.13.75.130:37633)
I20260923 15:01:39.929595 15851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:39.926485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.131:54305 (local address 127.13.75.129:36423)
0923 15:01:39.926562 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0923 15:01:39.926564 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:01:39.926606 (+ 42us) server_negotiation.cc:407] Connection header received
0923 15:01:39.926730 (+ 124us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:01:39.926731 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:01:39.926743 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:01:39.926801 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:01:39.927249 (+ 448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:39.927771 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:39.928849 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:39.929214 (+ 365us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:39.929304 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:01:39.929314 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0923 15:01:39.929464 (+ 150us) server_negotiation.cc:299] Negotiation successful
0923 15:01:39.929501 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260923 15:01:39.929584 15725 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:39.926462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.129:36423 (local address 127.13.75.131:54305)
0923 15:01:39.926547 (+ 85us) negotiation.cc:107] Waiting for socket to connect
0923 15:01:39.926562 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0923 15:01:39.926625 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:01:39.927119 (+ 494us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:01:39.927120 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:01:39.927121 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:01:39.927169 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:01:39.927170 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:01:39.927872 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:01:39.927873 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:01:39.928757 (+ 884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:01:39.928760 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:01:39.929304 (+ 544us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:01:39.929305 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:01:39.929382 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:01:39.929385 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 15:01:39.929454 (+ 69us) client_negotiation.cc:240] Negotiation successful
0923 15:01:39.929501 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":59,"mutex_wait_us":74}
I20260923 15:01:39.929670 15854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:39.926731 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.131:57783 (local address 127.13.75.130:37633)
0923 15:01:39.926938 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0923 15:01:39.926941 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:01:39.926951 (+ 10us) server_negotiation.cc:407] Connection header received
0923 15:01:39.926980 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:01:39.926981 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:01:39.926990 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:01:39.927070 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:01:39.927270 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:39.927776 (+ 506us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:39.928881 (+ 1105us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:39.929273 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:39.929371 (+ 98us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:01:39.929380 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0923 15:01:39.929552 (+ 172us) server_negotiation.cc:299] Negotiation successful
0923 15:01:39.929595 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":107}
I20260923 15:01:39.929714 15864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:39.926635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.130:37633 (local address 127.13.75.131:57783)
0923 15:01:39.926817 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0923 15:01:39.926828 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 15:01:39.926908 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:01:39.927076 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:01:39.927077 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:01:39.927080 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:01:39.927161 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:01:39.927163 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:01:39.927914 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:01:39.927915 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:01:39.928769 (+ 854us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:01:39.928772 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:01:39.929382 (+ 610us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:01:39.929383 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:01:39.929462 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:01:39.929466 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 15:01:39.929531 (+ 65us) client_negotiation.cc:240] Negotiation successful
0923 15:01:39.929575 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":162,"mutex_wait_us":2,"thread_start_us":109,"threads_started":1}
I20260923 15:01:39.930050 15497 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b7eb5458d9f3484d9e89d7cfd71fe5f3" candidate_uuid: "2cd60cb8df9646af83deb3452da89de2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e03dc9443b94cd09a97dc52085ffead" is_pre_election: true
I20260923 15:01:39.930218 15497 raft_consensus.cc:2454] T b7eb5458d9f3484d9e89d7cfd71fe5f3 P 6e03dc9443b94cd09a97dc52085ffead [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2cd60cb8df9646af83deb3452da89de2 in term 0.
I20260923 15:01:39.930267 15646 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b7eb5458d9f3484d9e89d7cfd71fe5f3" candidate_uuid: "2cd60cb8df9646af83deb3452da89de2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e1f390e2f6b43ef846a9189c50427e8" is_pre_election: true
I20260923 15:01:39.930464 15646 raft_consensus.cc:2454] T b7eb5458d9f3484d9e89d7cfd71fe5f3 P 3e1f390e2f6b43ef846a9189c50427e8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2cd60cb8df9646af83deb3452da89de2 in term 0.
I20260923 15:01:39.930480 15720 leader_election.cc:302] T b7eb5458d9f3484d9e89d7cfd71fe5f3 P 2cd60cb8df9646af83deb3452da89de2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2cd60cb8df9646af83deb3452da89de2, 6e03dc9443b94cd09a97dc52085ffead; no voters:
I20260923 15:01:39.930680 15861 raft_consensus.cc:2781] T b7eb5458d9f3484d9e89d7cfd71fe5f3 P 2cd60cb8df9646af83deb3452da89de2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 15:01:39.930738 15861 raft_consensus.cc:482] T b7eb5458d9f3484d9e89d7cfd71fe5f3 P 2cd60cb8df9646af83deb3452da89de2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 15:01:39.930756 15861 raft_consensus.cc:3037] T b7eb5458d9f3484d9e89d7cfd71fe5f3 P 2cd60cb8df9646af83deb3452da89de2 [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:01:39.931543 15861 raft_consensus.cc:504] T b7eb5458d9f3484d9e89d7cfd71fe5f3 P 2cd60cb8df9646af83deb3452da89de2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e03dc9443b94cd09a97dc52085ffead" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 36423 } } peers { permanent_uuid: "3e1f390e2f6b43ef846a9189c50427e8" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 37633 } } peers { permanent_uuid: "2cd60cb8df9646af83deb3452da89de2" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 43295 } }
I20260923 15:01:39.931691 15861 leader_election.cc:288] T b7eb5458d9f3484d9e89d7cfd71fe5f3 P 2cd60cb8df9646af83deb3452da89de2 [CANDIDATE]: Term 1 election: Requested vote from peers 6e03dc9443b94cd09a97dc52085ffead (127.13.75.129:36423), 3e1f390e2f6b43ef846a9189c50427e8 (127.13.75.130:37633)
I20260923 15:01:39.931902 15497 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b7eb5458d9f3484d9e89d7cfd71fe5f3" candidate_uuid: "2cd60cb8df9646af83deb3452da89de2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e03dc9443b94cd09a97dc52085ffead"
I20260923 15:01:39.931995 15497 raft_consensus.cc:3037] T b7eb5458d9f3484d9e89d7cfd71fe5f3 P 6e03dc9443b94cd09a97dc52085ffead [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:01:39.932878 15497 raft_consensus.cc:2454] T b7eb5458d9f3484d9e89d7cfd71fe5f3 P 6e03dc9443b94cd09a97dc52085ffead [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2cd60cb8df9646af83deb3452da89de2 in term 1.
I20260923 15:01:39.933032 15646 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b7eb5458d9f3484d9e89d7cfd71fe5f3" candidate_uuid: "2cd60cb8df9646af83deb3452da89de2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e1f390e2f6b43ef846a9189c50427e8"
I20260923 15:01:39.933075 15720 leader_election.cc:302] T b7eb5458d9f3484d9e89d7cfd71fe5f3 P 2cd60cb8df9646af83deb3452da89de2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2cd60cb8df9646af83deb3452da89de2, 6e03dc9443b94cd09a97dc52085ffead; no voters:
I20260923 15:01:39.933135 15646 raft_consensus.cc:3037] T b7eb5458d9f3484d9e89d7cfd71fe5f3 P 3e1f390e2f6b43ef846a9189c50427e8 [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:01:39.933175 15861 raft_consensus.cc:2781] T b7eb5458d9f3484d9e89d7cfd71fe5f3 P 2cd60cb8df9646af83deb3452da89de2 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 15:01:39.933367 15861 raft_consensus.cc:686] T b7eb5458d9f3484d9e89d7cfd71fe5f3 P 2cd60cb8df9646af83deb3452da89de2 [term 1 LEADER]: Becoming Leader. State: Replica: 2cd60cb8df9646af83deb3452da89de2, State: Running, Role: LEADER
I20260923 15:01:39.933481 15861 consensus_queue.cc:237] T b7eb5458d9f3484d9e89d7cfd71fe5f3 P 2cd60cb8df9646af83deb3452da89de2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e03dc9443b94cd09a97dc52085ffead" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 36423 } } peers { permanent_uuid: "3e1f390e2f6b43ef846a9189c50427e8" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 37633 } } peers { permanent_uuid: "2cd60cb8df9646af83deb3452da89de2" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 43295 } }
I20260923 15:01:39.934195 15646 raft_consensus.cc:2454] T b7eb5458d9f3484d9e89d7cfd71fe5f3 P 3e1f390e2f6b43ef846a9189c50427e8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2cd60cb8df9646af83deb3452da89de2 in term 1.
I20260923 15:01:39.934526 15349 catalog_manager.cc:5495] T b7eb5458d9f3484d9e89d7cfd71fe5f3 P 2cd60cb8df9646af83deb3452da89de2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2cd60cb8df9646af83deb3452da89de2 (127.13.75.131). New cstate: current_term: 1 leader_uuid: "2cd60cb8df9646af83deb3452da89de2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e03dc9443b94cd09a97dc52085ffead" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 36423 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e1f390e2f6b43ef846a9189c50427e8" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 37633 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2cd60cb8df9646af83deb3452da89de2" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 43295 } health_report { overall_health: HEALTHY } } }
Sep 23 15:01:39 dist-test-slave-cnb0 krb5kdc[15304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175699, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.190@KRBTEST.COM for kudu/127.13.75.190@KRBTEST.COM
I20260923 15:01:39.995589 15872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:39.987850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:43703 (local address 127.0.0.1:46012)
0923 15:01:39.988075 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0923 15:01:39.988104 (+ 29us) client_negotiation.cc:174] Beginning negotiation
0923 15:01:39.988177 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:01:39.988345 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:01:39.988347 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:01:39.988701 (+ 354us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:01:39.988751 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:01:39.988753 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:01:39.989409 (+ 656us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:01:39.989410 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:01:39.989997 (+ 587us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:01:39.989999 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:01:39.990354 (+ 355us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:01:39.990355 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:01:39.990400 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:01:39.991041 (+ 641us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:01:39.991046 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:01:39.992542 (+ 1496us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:01:39.994236 (+ 1694us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:01:39.994245 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:01:39.994247 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:01:39.994487 (+ 240us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:01:39.994817 (+ 330us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:01:39.994818 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:01:39.994819 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:01:39.994870 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:01:39.995256 (+ 386us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:01:39.995263 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:01:39.995335 (+ 72us) client_negotiation.cc:769] Sending connection context
0923 15:01:39.995404 (+ 69us) client_negotiation.cc:240] Negotiation successful
0923 15:01:39.995510 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":135,"threads_started":1}
I20260923 15:01:39.995702 15399 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:39.987918 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46012 (local address 127.13.75.190:43703)
0923 15:01:39.987988 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0923 15:01:39.987992 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 15:01:39.988191 (+ 199us) server_negotiation.cc:407] Connection header received
0923 15:01:39.988232 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:01:39.988233 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:01:39.988244 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:01:39.988304 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:01:39.988833 (+ 529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:39.989330 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:39.990101 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:39.990284 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:39.990347 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:01:39.992612 (+ 2265us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:01:39.992613 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:01:39.992614 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:01:39.992625 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:01:39.994121 (+ 1496us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:01:39.994556 (+ 435us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:01:39.994557 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:01:39.994557 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:01:39.994611 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:01:39.994997 (+ 386us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:01:39.994998 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:01:39.994998 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:01:39.995173 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:01:39.995244 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0923 15:01:39.995446 (+ 202us) server_negotiation.cc:299] Negotiation successful
0923 15:01:39.995572 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":45,"server-negotiator.queue_time_us":40}
I20260923 15:01:39.998438 15349 catalog_manager.cc:1908] Servicing CreateTable request from {username='kudu', principal='kudu/127.13.75.190@KRBTEST.COM'} at 127.0.0.1:46012:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260923 15:01:39.998695 15349 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 15:01:40.000640 15471 tablet_service.cc:1467] Processing CreateTablet for tablet 198f11fe784349c19da6e9f63ea90bef (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=259a93ca625d4b79a7e694f8b2623942]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260923 15:01:40.000658 15766 tablet_service.cc:1467] Processing CreateTablet for tablet 198f11fe784349c19da6e9f63ea90bef (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=259a93ca625d4b79a7e694f8b2623942]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260923 15:01:40.000682 15626 tablet_service.cc:1467] Processing CreateTablet for tablet 198f11fe784349c19da6e9f63ea90bef (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=259a93ca625d4b79a7e694f8b2623942]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260923 15:01:40.000839 15766 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 198f11fe784349c19da6e9f63ea90bef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:01:40.000839 15626 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 198f11fe784349c19da6e9f63ea90bef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:01:40.000900 15471 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 198f11fe784349c19da6e9f63ea90bef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:01:40.002266 15857 tablet_bootstrap.cc:492] T 198f11fe784349c19da6e9f63ea90bef P 6e03dc9443b94cd09a97dc52085ffead: Bootstrap starting.
I20260923 15:01:40.002414 15855 tablet_bootstrap.cc:492] T 198f11fe784349c19da6e9f63ea90bef P 2cd60cb8df9646af83deb3452da89de2: Bootstrap starting.
I20260923 15:01:40.002414 15856 tablet_bootstrap.cc:492] T 198f11fe784349c19da6e9f63ea90bef P 3e1f390e2f6b43ef846a9189c50427e8: Bootstrap starting.
I20260923 15:01:40.002682 15857 tablet_bootstrap.cc:654] T 198f11fe784349c19da6e9f63ea90bef P 6e03dc9443b94cd09a97dc52085ffead: Neither blocks nor log segments found. Creating new log.
I20260923 15:01:40.003083 15855 tablet_bootstrap.cc:654] T 198f11fe784349c19da6e9f63ea90bef P 2cd60cb8df9646af83deb3452da89de2: Neither blocks nor log segments found. Creating new log.
I20260923 15:01:40.003189 15856 tablet_bootstrap.cc:654] T 198f11fe784349c19da6e9f63ea90bef P 3e1f390e2f6b43ef846a9189c50427e8: Neither blocks nor log segments found. Creating new log.
I20260923 15:01:40.003944 15855 tablet_bootstrap.cc:492] T 198f11fe784349c19da6e9f63ea90bef P 2cd60cb8df9646af83deb3452da89de2: No bootstrap required, opened a new log
I20260923 15:01:40.003944 15856 tablet_bootstrap.cc:492] T 198f11fe784349c19da6e9f63ea90bef P 3e1f390e2f6b43ef846a9189c50427e8: No bootstrap required, opened a new log
I20260923 15:01:40.004004 15856 ts_tablet_manager.cc:1397] T 198f11fe784349c19da6e9f63ea90bef P 3e1f390e2f6b43ef846a9189c50427e8: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 15:01:40.004004 15855 ts_tablet_manager.cc:1397] T 198f11fe784349c19da6e9f63ea90bef P 2cd60cb8df9646af83deb3452da89de2: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 15:01:40.004191 15855 raft_consensus.cc:348] T 198f11fe784349c19da6e9f63ea90bef P 2cd60cb8df9646af83deb3452da89de2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e03dc9443b94cd09a97dc52085ffead" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 36423 } } peers { permanent_uuid: "3e1f390e2f6b43ef846a9189c50427e8" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 37633 } } peers { permanent_uuid: "2cd60cb8df9646af83deb3452da89de2" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 43295 } }
I20260923 15:01:40.004194 15856 raft_consensus.cc:348] T 198f11fe784349c19da6e9f63ea90bef P 3e1f390e2f6b43ef846a9189c50427e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e03dc9443b94cd09a97dc52085ffead" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 36423 } } peers { permanent_uuid: "3e1f390e2f6b43ef846a9189c50427e8" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 37633 } } peers { permanent_uuid: "2cd60cb8df9646af83deb3452da89de2" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 43295 } }
I20260923 15:01:40.004276 15856 raft_consensus.cc:374] T 198f11fe784349c19da6e9f63ea90bef P 3e1f390e2f6b43ef846a9189c50427e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:01:40.004277 15855 raft_consensus.cc:374] T 198f11fe784349c19da6e9f63ea90bef P 2cd60cb8df9646af83deb3452da89de2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:01:40.004300 15856 raft_consensus.cc:729] T 198f11fe784349c19da6e9f63ea90bef P 3e1f390e2f6b43ef846a9189c50427e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e1f390e2f6b43ef846a9189c50427e8, State: Initialized, Role: FOLLOWER
I20260923 15:01:40.004300 15855 raft_consensus.cc:729] T 198f11fe784349c19da6e9f63ea90bef P 2cd60cb8df9646af83deb3452da89de2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2cd60cb8df9646af83deb3452da89de2, State: Initialized, Role: FOLLOWER
I20260923 15:01:40.004349 15856 consensus_queue.cc:260] T 198f11fe784349c19da6e9f63ea90bef P 3e1f390e2f6b43ef846a9189c50427e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e03dc9443b94cd09a97dc52085ffead" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 36423 } } peers { permanent_uuid: "3e1f390e2f6b43ef846a9189c50427e8" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 37633 } } peers { permanent_uuid: "2cd60cb8df9646af83deb3452da89de2" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 43295 } }
I20260923 15:01:40.004349 15855 consensus_queue.cc:260] T 198f11fe784349c19da6e9f63ea90bef P 2cd60cb8df9646af83deb3452da89de2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e03dc9443b94cd09a97dc52085ffead" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 36423 } } peers { permanent_uuid: "3e1f390e2f6b43ef846a9189c50427e8" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 37633 } } peers { permanent_uuid: "2cd60cb8df9646af83deb3452da89de2" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 43295 } }
I20260923 15:01:40.004446 15856 ts_tablet_manager.cc:1428] T 198f11fe784349c19da6e9f63ea90bef P 3e1f390e2f6b43ef846a9189c50427e8: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260923 15:01:40.004458 15855 ts_tablet_manager.cc:1428] T 198f11fe784349c19da6e9f63ea90bef P 2cd60cb8df9646af83deb3452da89de2: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260923 15:01:40.004539 15862 tablet_replica.cc:439] T 198f11fe784349c19da6e9f63ea90bef P 3e1f390e2f6b43ef846a9189c50427e8: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e03dc9443b94cd09a97dc52085ffead" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 36423 } } peers { permanent_uuid: "3e1f390e2f6b43ef846a9189c50427e8" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 37633 } } peers { permanent_uuid: "2cd60cb8df9646af83deb3452da89de2" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 43295 } } }
I20260923 15:01:40.004670 15862 tablet_replica.cc:442] T 198f11fe784349c19da6e9f63ea90bef P 3e1f390e2f6b43ef846a9189c50427e8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260923 15:01:40.004681 15857 tablet_bootstrap.cc:492] T 198f11fe784349c19da6e9f63ea90bef P 6e03dc9443b94cd09a97dc52085ffead: No bootstrap required, opened a new log
I20260923 15:01:40.004734 15857 ts_tablet_manager.cc:1397] T 198f11fe784349c19da6e9f63ea90bef P 6e03dc9443b94cd09a97dc52085ffead: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 15:01:40.004818 15861 tablet_replica.cc:439] T 198f11fe784349c19da6e9f63ea90bef P 2cd60cb8df9646af83deb3452da89de2: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e03dc9443b94cd09a97dc52085ffead" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 36423 } } peers { permanent_uuid: "3e1f390e2f6b43ef846a9189c50427e8" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 37633 } } peers { permanent_uuid: "2cd60cb8df9646af83deb3452da89de2" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 43295 } } }
I20260923 15:01:40.004901 15857 raft_consensus.cc:348] T 198f11fe784349c19da6e9f63ea90bef P 6e03dc9443b94cd09a97dc52085ffead [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e03dc9443b94cd09a97dc52085ffead" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 36423 } } peers { permanent_uuid: "3e1f390e2f6b43ef846a9189c50427e8" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 37633 } } peers { permanent_uuid: "2cd60cb8df9646af83deb3452da89de2" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 43295 } }
I20260923 15:01:40.004935 15861 tablet_replica.cc:442] T 198f11fe784349c19da6e9f63ea90bef P 2cd60cb8df9646af83deb3452da89de2: This TxnStatusTablet replica's current role is: FOLLOWER
I20260923 15:01:40.004990 15857 raft_consensus.cc:374] T 198f11fe784349c19da6e9f63ea90bef P 6e03dc9443b94cd09a97dc52085ffead [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:01:40.005016 15857 raft_consensus.cc:729] T 198f11fe784349c19da6e9f63ea90bef P 6e03dc9443b94cd09a97dc52085ffead [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e03dc9443b94cd09a97dc52085ffead, State: Initialized, Role: FOLLOWER
I20260923 15:01:40.005076 15857 consensus_queue.cc:260] T 198f11fe784349c19da6e9f63ea90bef P 6e03dc9443b94cd09a97dc52085ffead [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e03dc9443b94cd09a97dc52085ffead" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 36423 } } peers { permanent_uuid: "3e1f390e2f6b43ef846a9189c50427e8" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 37633 } } peers { permanent_uuid: "2cd60cb8df9646af83deb3452da89de2" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 43295 } }
I20260923 15:01:40.005239 15857 ts_tablet_manager.cc:1428] T 198f11fe784349c19da6e9f63ea90bef P 6e03dc9443b94cd09a97dc52085ffead: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260923 15:01:40.005331 15863 tablet_replica.cc:439] T 198f11fe784349c19da6e9f63ea90bef P 6e03dc9443b94cd09a97dc52085ffead: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e03dc9443b94cd09a97dc52085ffead" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 36423 } } peers { permanent_uuid: "3e1f390e2f6b43ef846a9189c50427e8" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 37633 } } peers { permanent_uuid: "2cd60cb8df9646af83deb3452da89de2" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 43295 } } }
I20260923 15:01:40.005442 15863 tablet_replica.cc:442] T 198f11fe784349c19da6e9f63ea90bef P 6e03dc9443b94cd09a97dc52085ffead: This TxnStatusTablet replica's current role is: FOLLOWER
W20260923 15:01:40.021692 15833 tablet.cc:2369] T 198f11fe784349c19da6e9f63ea90bef P 2cd60cb8df9646af83deb3452da89de2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 15:01:40.094247 15863 raft_consensus.cc:482] T 198f11fe784349c19da6e9f63ea90bef P 6e03dc9443b94cd09a97dc52085ffead [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 15:01:40.094363 15863 raft_consensus.cc:504] T 198f11fe784349c19da6e9f63ea90bef P 6e03dc9443b94cd09a97dc52085ffead [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e03dc9443b94cd09a97dc52085ffead" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 36423 } } peers { permanent_uuid: "3e1f390e2f6b43ef846a9189c50427e8" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 37633 } } peers { permanent_uuid: "2cd60cb8df9646af83deb3452da89de2" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 43295 } }
I20260923 15:01:40.094740 15863 leader_election.cc:288] T 198f11fe784349c19da6e9f63ea90bef P 6e03dc9443b94cd09a97dc52085ffead [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3e1f390e2f6b43ef846a9189c50427e8 (127.13.75.130:37633), 2cd60cb8df9646af83deb3452da89de2 (127.13.75.131:43295)
I20260923 15:01:40.099256 15854 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:40.095023 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.129:58373 (local address 127.13.75.130:37633)
0923 15:01:40.095088 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0923 15:01:40.095091 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:01:40.095231 (+ 140us) server_negotiation.cc:407] Connection header received
0923 15:01:40.095276 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:01:40.095277 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:01:40.095289 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:01:40.095379 (+ 90us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:01:40.095678 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:40.096372 (+ 694us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:40.098186 (+ 1814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:40.098737 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:40.098820 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:01:40.098829 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0923 15:01:40.099106 (+ 277us) server_negotiation.cc:299] Negotiation successful
0923 15:01:40.099184 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260923 15:01:40.099367 15877 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:40.094970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.130:37633 (local address 127.13.75.129:58373)
0923 15:01:40.095132 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0923 15:01:40.095145 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0923 15:01:40.095219 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:01:40.095454 (+ 235us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:01:40.095455 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:01:40.095458 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:01:40.095603 (+ 145us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:01:40.095606 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:01:40.096490 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:01:40.096491 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:01:40.098072 (+ 1581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:01:40.098076 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:01:40.098905 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:01:40.098906 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:01:40.099010 (+ 104us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:01:40.099017 (+ 7us) client_negotiation.cc:769] Sending connection context
0923 15:01:40.099080 (+ 63us) client_negotiation.cc:240] Negotiation successful
0923 15:01:40.099164 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"mutex_wait_us":125,"spinlock_wait_cycles":1280,"thread_start_us":104,"threads_started":1}
I20260923 15:01:40.099666 15646 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "198f11fe784349c19da6e9f63ea90bef" candidate_uuid: "6e03dc9443b94cd09a97dc52085ffead" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e1f390e2f6b43ef846a9189c50427e8" is_pre_election: true
I20260923 15:01:40.099761 15646 raft_consensus.cc:2454] T 198f11fe784349c19da6e9f63ea90bef P 3e1f390e2f6b43ef846a9189c50427e8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e03dc9443b94cd09a97dc52085ffead in term 0.
I20260923 15:01:40.099987 15440 leader_election.cc:302] T 198f11fe784349c19da6e9f63ea90bef P 6e03dc9443b94cd09a97dc52085ffead [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e1f390e2f6b43ef846a9189c50427e8, 6e03dc9443b94cd09a97dc52085ffead; no voters:
I20260923 15:01:40.100162 15863 raft_consensus.cc:2781] T 198f11fe784349c19da6e9f63ea90bef P 6e03dc9443b94cd09a97dc52085ffead [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 15:01:40.100219 15863 raft_consensus.cc:482] T 198f11fe784349c19da6e9f63ea90bef P 6e03dc9443b94cd09a97dc52085ffead [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 15:01:40.100246 15863 raft_consensus.cc:3037] T 198f11fe784349c19da6e9f63ea90bef P 6e03dc9443b94cd09a97dc52085ffead [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:01:40.101140 15863 raft_consensus.cc:504] T 198f11fe784349c19da6e9f63ea90bef P 6e03dc9443b94cd09a97dc52085ffead [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e03dc9443b94cd09a97dc52085ffead" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 36423 } } peers { permanent_uuid: "3e1f390e2f6b43ef846a9189c50427e8" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 37633 } } peers { permanent_uuid: "2cd60cb8df9646af83deb3452da89de2" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 43295 } }
I20260923 15:01:40.101311 15863 leader_election.cc:288] T 198f11fe784349c19da6e9f63ea90bef P 6e03dc9443b94cd09a97dc52085ffead [CANDIDATE]: Term 1 election: Requested vote from peers 3e1f390e2f6b43ef846a9189c50427e8 (127.13.75.130:37633), 2cd60cb8df9646af83deb3452da89de2 (127.13.75.131:43295)
I20260923 15:01:40.101459 15646 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "198f11fe784349c19da6e9f63ea90bef" candidate_uuid: "6e03dc9443b94cd09a97dc52085ffead" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e1f390e2f6b43ef846a9189c50427e8"
I20260923 15:01:40.101544 15646 raft_consensus.cc:3037] T 198f11fe784349c19da6e9f63ea90bef P 3e1f390e2f6b43ef846a9189c50427e8 [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:01:40.101555 15876 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:40.094962 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.131:43295 (local address 127.13.75.129:52647)
0923 15:01:40.095132 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0923 15:01:40.095148 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0923 15:01:40.095219 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:01:40.095440 (+ 221us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:01:40.095441 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:01:40.095443 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:01:40.095566 (+ 123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:01:40.095568 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:01:40.096489 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:01:40.096491 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:01:40.098072 (+ 1581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:01:40.098076 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:01:40.098876 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:01:40.098903 (+ 27us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:01:40.101323 (+ 2420us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:01:40.101331 (+ 8us) client_negotiation.cc:769] Sending connection context
0923 15:01:40.101414 (+ 83us) client_negotiation.cc:240] Negotiation successful
0923 15:01:40.101474 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"mutex_wait_us":94,"thread_start_us":85,"threads_started":1}
I20260923 15:01:40.101553 15852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:40.095025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.129:52647 (local address 127.13.75.131:43295)
0923 15:01:40.095096 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0923 15:01:40.095098 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:01:40.095221 (+ 123us) server_negotiation.cc:407] Connection header received
0923 15:01:40.095344 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:01:40.095345 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:01:40.095356 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:01:40.095420 (+ 64us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:01:40.095627 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:40.096390 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:40.098185 (+ 1795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:40.098747 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:40.098820 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:01:40.098829 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0923 15:01:40.101435 (+ 2606us) server_negotiation.cc:299] Negotiation successful
0923 15:01:40.101491 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260923 15:01:40.102089 15786 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "198f11fe784349c19da6e9f63ea90bef" candidate_uuid: "6e03dc9443b94cd09a97dc52085ffead" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2cd60cb8df9646af83deb3452da89de2" is_pre_election: true
I20260923 15:01:40.102093 15785 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "198f11fe784349c19da6e9f63ea90bef" candidate_uuid: "6e03dc9443b94cd09a97dc52085ffead" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2cd60cb8df9646af83deb3452da89de2"
I20260923 15:01:40.102240 15786 raft_consensus.cc:2454] T 198f11fe784349c19da6e9f63ea90bef P 2cd60cb8df9646af83deb3452da89de2 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 6e03dc9443b94cd09a97dc52085ffead in term 0.
I20260923 15:01:40.102401 15646 raft_consensus.cc:2454] T 198f11fe784349c19da6e9f63ea90bef P 3e1f390e2f6b43ef846a9189c50427e8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 6e03dc9443b94cd09a97dc52085ffead in term 1.
I20260923 15:01:40.102564 15440 leader_election.cc:302] T 198f11fe784349c19da6e9f63ea90bef P 6e03dc9443b94cd09a97dc52085ffead [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 3e1f390e2f6b43ef846a9189c50427e8, 6e03dc9443b94cd09a97dc52085ffead; no voters:
I20260923 15:01:40.102711 15863 raft_consensus.cc:2781] T 198f11fe784349c19da6e9f63ea90bef P 6e03dc9443b94cd09a97dc52085ffead [term 1 FOLLOWER]: Leader election won for term 1
I20260923 15:01:40.102854 15863 raft_consensus.cc:686] T 198f11fe784349c19da6e9f63ea90bef P 6e03dc9443b94cd09a97dc52085ffead [term 1 LEADER]: Becoming Leader. State: Replica: 6e03dc9443b94cd09a97dc52085ffead, State: Running, Role: LEADER
I20260923 15:01:40.103027 15863 consensus_queue.cc:237] T 198f11fe784349c19da6e9f63ea90bef P 6e03dc9443b94cd09a97dc52085ffead [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e03dc9443b94cd09a97dc52085ffead" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 36423 } } peers { permanent_uuid: "3e1f390e2f6b43ef846a9189c50427e8" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 37633 } } peers { permanent_uuid: "2cd60cb8df9646af83deb3452da89de2" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 43295 } }
I20260923 15:01:40.103716 15878 tablet_replica.cc:439] T 198f11fe784349c19da6e9f63ea90bef P 6e03dc9443b94cd09a97dc52085ffead: TxnStatusTablet state changed. Reason: New leader 6e03dc9443b94cd09a97dc52085ffead. Latest consensus state: current_term: 1 leader_uuid: "6e03dc9443b94cd09a97dc52085ffead" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e03dc9443b94cd09a97dc52085ffead" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 36423 } } peers { permanent_uuid: "3e1f390e2f6b43ef846a9189c50427e8" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 37633 } } peers { permanent_uuid: "2cd60cb8df9646af83deb3452da89de2" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 43295 } } }
I20260923 15:01:40.103834 15878 tablet_replica.cc:442] T 198f11fe784349c19da6e9f63ea90bef P 6e03dc9443b94cd09a97dc52085ffead: This TxnStatusTablet replica's current role is: LEADER
I20260923 15:01:40.104054 15880 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260923 15:01:40.104208 15880 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260923 15:01:40.104184 15349 catalog_manager.cc:5495] T 198f11fe784349c19da6e9f63ea90bef P 6e03dc9443b94cd09a97dc52085ffead reported cstate change: term changed from 0 to 1, leader changed from <none> to 6e03dc9443b94cd09a97dc52085ffead (127.13.75.129). New cstate: current_term: 1 leader_uuid: "6e03dc9443b94cd09a97dc52085ffead" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e03dc9443b94cd09a97dc52085ffead" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 36423 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "3e1f390e2f6b43ef846a9189c50427e8" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 37633 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2cd60cb8df9646af83deb3452da89de2" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 43295 } health_report { overall_health: UNKNOWN } } }
I20260923 15:01:40.125965 15872 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:40.122477 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.129:36423 (local address 127.0.0.1:40182)
0923 15:01:40.122567 (+ 90us) negotiation.cc:107] Waiting for socket to connect
0923 15:01:40.122581 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0923 15:01:40.122643 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:01:40.122857 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:01:40.122857 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:01:40.122859 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0923 15:01:40.122993 (+ 134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:01:40.122995 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:01:40.124040 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:01:40.124041 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:01:40.124693 (+ 652us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:01:40.124696 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:01:40.125247 (+ 551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:01:40.125248 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:01:40.125322 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:01:40.125342 (+ 20us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0923 15:01:40.125732 (+ 390us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0923 15:01:40.125748 (+ 16us) client_negotiation.cc:769] Sending connection context
0923 15:01:40.125833 (+ 85us) client_negotiation.cc:240] Negotiation successful
0923 15:01:40.125881 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44}
I20260923 15:01:40.125960 15851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:40.122584 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40182 (local address 127.13.75.129:36423)
0923 15:01:40.122655 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0923 15:01:40.122657 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:01:40.122669 (+ 12us) server_negotiation.cc:407] Connection header received
0923 15:01:40.122737 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:01:40.122738 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:01:40.122762 (+ 24us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:01:40.122854 (+ 92us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 15:01:40.123066 (+ 212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:40.123933 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:40.124784 (+ 851us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:40.125139 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:40.125218 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:01:40.125464 (+ 246us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 15:01:40.125640 (+ 176us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 15:01:40.125726 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0923 15:01:40.125848 (+ 122us) server_negotiation.cc:299] Negotiation successful
0923 15:01:40.125872 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260923 15:01:40.127925 15785 raft_consensus.cc:3037] T 198f11fe784349c19da6e9f63ea90bef P 2cd60cb8df9646af83deb3452da89de2 [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:01:40.128147 15646 raft_consensus.cc:1261] T 198f11fe784349c19da6e9f63ea90bef P 3e1f390e2f6b43ef846a9189c50427e8 [term 1 FOLLOWER]: Refusing update from remote peer 6e03dc9443b94cd09a97dc52085ffead: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 15:01:40.128453 15878 consensus_queue.cc:1035] T 198f11fe784349c19da6e9f63ea90bef P 6e03dc9443b94cd09a97dc52085ffead [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e1f390e2f6b43ef846a9189c50427e8" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 37633 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 15:01:40.128955 15785 raft_consensus.cc:1261] T 198f11fe784349c19da6e9f63ea90bef P 2cd60cb8df9646af83deb3452da89de2 [term 1 FOLLOWER]: Refusing update from remote peer 6e03dc9443b94cd09a97dc52085ffead: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 15:01:40.129213 15878 consensus_queue.cc:1035] T 198f11fe784349c19da6e9f63ea90bef P 6e03dc9443b94cd09a97dc52085ffead [LEADER]: Connected to new peer: Peer: permanent_uuid: "2cd60cb8df9646af83deb3452da89de2" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 43295 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 15:01:40.129804 15862 tablet_replica.cc:439] T 198f11fe784349c19da6e9f63ea90bef P 3e1f390e2f6b43ef846a9189c50427e8: TxnStatusTablet state changed. Reason: New leader 6e03dc9443b94cd09a97dc52085ffead. Latest consensus state: current_term: 1 leader_uuid: "6e03dc9443b94cd09a97dc52085ffead" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e03dc9443b94cd09a97dc52085ffead" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 36423 } } peers { permanent_uuid: "3e1f390e2f6b43ef846a9189c50427e8" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 37633 } } peers { permanent_uuid: "2cd60cb8df9646af83deb3452da89de2" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 43295 } } }
I20260923 15:01:40.129938 15862 tablet_replica.cc:442] T 198f11fe784349c19da6e9f63ea90bef P 3e1f390e2f6b43ef846a9189c50427e8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260923 15:01:40.130447 15861 tablet_replica.cc:439] T 198f11fe784349c19da6e9f63ea90bef P 2cd60cb8df9646af83deb3452da89de2: TxnStatusTablet state changed. Reason: New leader 6e03dc9443b94cd09a97dc52085ffead. Latest consensus state: current_term: 1 leader_uuid: "6e03dc9443b94cd09a97dc52085ffead" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e03dc9443b94cd09a97dc52085ffead" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 36423 } } peers { permanent_uuid: "3e1f390e2f6b43ef846a9189c50427e8" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 37633 } } peers { permanent_uuid: "2cd60cb8df9646af83deb3452da89de2" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 43295 } } }
I20260923 15:01:40.130542 15861 tablet_replica.cc:442] T 198f11fe784349c19da6e9f63ea90bef P 2cd60cb8df9646af83deb3452da89de2: This TxnStatusTablet replica's current role is: FOLLOWER
I20260923 15:01:40.131135 15878 tablet_replica.cc:439] T 198f11fe784349c19da6e9f63ea90bef P 6e03dc9443b94cd09a97dc52085ffead: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6e03dc9443b94cd09a97dc52085ffead" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e03dc9443b94cd09a97dc52085ffead" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 36423 } } peers { permanent_uuid: "3e1f390e2f6b43ef846a9189c50427e8" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 37633 } } peers { permanent_uuid: "2cd60cb8df9646af83deb3452da89de2" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 43295 } } }
I20260923 15:01:40.131273 15878 tablet_replica.cc:442] T 198f11fe784349c19da6e9f63ea90bef P 6e03dc9443b94cd09a97dc52085ffead: This TxnStatusTablet replica's current role is: LEADER
I20260923 15:01:40.131800 15863 tablet_replica.cc:439] T 198f11fe784349c19da6e9f63ea90bef P 6e03dc9443b94cd09a97dc52085ffead: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "6e03dc9443b94cd09a97dc52085ffead" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e03dc9443b94cd09a97dc52085ffead" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 36423 } } peers { permanent_uuid: "3e1f390e2f6b43ef846a9189c50427e8" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 37633 } } peers { permanent_uuid: "2cd60cb8df9646af83deb3452da89de2" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 43295 } } }
I20260923 15:01:40.131894 15863 tablet_replica.cc:442] T 198f11fe784349c19da6e9f63ea90bef P 6e03dc9443b94cd09a97dc52085ffead: This TxnStatusTablet replica's current role is: LEADER
I20260923 15:01:40.131917 15884 mvcc.cc:205] Tried to move back new op lower bound from 7332559667721416704 to 7332559667622592512. Current Snapshot: MvccSnapshot[applied={T|T < 7332559667721416704}]
I20260923 15:01:40.132037 15862 tablet_replica.cc:439] T 198f11fe784349c19da6e9f63ea90bef P 3e1f390e2f6b43ef846a9189c50427e8: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6e03dc9443b94cd09a97dc52085ffead" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e03dc9443b94cd09a97dc52085ffead" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 36423 } } peers { permanent_uuid: "3e1f390e2f6b43ef846a9189c50427e8" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 37633 } } peers { permanent_uuid: "2cd60cb8df9646af83deb3452da89de2" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 43295 } } }
I20260923 15:01:40.132118 15862 tablet_replica.cc:442] T 198f11fe784349c19da6e9f63ea90bef P 3e1f390e2f6b43ef846a9189c50427e8: This TxnStatusTablet replica's current role is: FOLLOWER
I20260923 15:01:40.132755 15861 tablet_replica.cc:439] T 198f11fe784349c19da6e9f63ea90bef P 2cd60cb8df9646af83deb3452da89de2: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "6e03dc9443b94cd09a97dc52085ffead" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e03dc9443b94cd09a97dc52085ffead" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 36423 } } peers { permanent_uuid: "3e1f390e2f6b43ef846a9189c50427e8" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 37633 } } peers { permanent_uuid: "2cd60cb8df9646af83deb3452da89de2" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 43295 } } }
I20260923 15:01:40.132844 15861 tablet_replica.cc:442] T 198f11fe784349c19da6e9f63ea90bef P 2cd60cb8df9646af83deb3452da89de2: This TxnStatusTablet replica's current role is: FOLLOWER
I20260923 15:01:40.136967 15852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:40.134138 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55728 (local address 127.13.75.131:43295)
0923 15:01:40.134186 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0923 15:01:40.134188 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:01:40.134198 (+ 10us) server_negotiation.cc:407] Connection header received
0923 15:01:40.134291 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:01:40.134292 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:01:40.134311 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:01:40.134493 (+ 182us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 15:01:40.134684 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:40.135286 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:40.136110 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:40.136334 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:40.136430 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:01:40.136623 (+ 193us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 15:01:40.136746 (+ 123us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 15:01:40.136833 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0923 15:01:40.136858 (+ 25us) server_negotiation.cc:299] Negotiation successful
0923 15:01:40.136880 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260923 15:01:40.142773 15851 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:40.139819 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.131:47641 (local address 127.13.75.129:36423)
0923 15:01:40.139893 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0923 15:01:40.139895 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:01:40.139904 (+ 9us) server_negotiation.cc:407] Connection header received
0923 15:01:40.139925 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:01:40.139926 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:01:40.139935 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:01:40.139994 (+ 59us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 15:01:40.140170 (+ 176us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:40.140652 (+ 482us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:40.141934 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:40.142115 (+ 181us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:40.142169 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:01:40.142434 (+ 265us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 15:01:40.142466 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 15:01:40.142610 (+ 144us) server_negotiation.cc:1035] Waiting for connection context
0923 15:01:40.142681 (+ 71us) server_negotiation.cc:299] Negotiation successful
0923 15:01:40.142704 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260923 15:01:40.142825 15838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:40.139561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.129:36423 (local address 127.13.75.131:47641)
0923 15:01:40.139640 (+ 79us) negotiation.cc:107] Waiting for socket to connect
0923 15:01:40.139654 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0923 15:01:40.139706 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:01:40.140019 (+ 313us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:01:40.140020 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:01:40.140022 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0923 15:01:40.140088 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:01:40.140090 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:01:40.140742 (+ 652us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:01:40.140743 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:01:40.141765 (+ 1022us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:01:40.141768 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:01:40.142190 (+ 422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:01:40.142191 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:01:40.142270 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:01:40.142290 (+ 20us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0923 15:01:40.142623 (+ 333us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0923 15:01:40.142625 (+ 2us) client_negotiation.cc:769] Sending connection context
0923 15:01:40.142675 (+ 50us) client_negotiation.cc:240] Negotiation successful
0923 15:01:40.142737 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260923 15:01:40.145592 15646 raft_consensus.cc:1261] T b7eb5458d9f3484d9e89d7cfd71fe5f3 P 3e1f390e2f6b43ef846a9189c50427e8 [term 1 FOLLOWER]: Refusing update from remote peer 2cd60cb8df9646af83deb3452da89de2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 15:01:40.145740 15497 raft_consensus.cc:1261] T b7eb5458d9f3484d9e89d7cfd71fe5f3 P 6e03dc9443b94cd09a97dc52085ffead [term 1 FOLLOWER]: Refusing update from remote peer 2cd60cb8df9646af83deb3452da89de2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 15:01:40.145879 15861 consensus_queue.cc:1035] T b7eb5458d9f3484d9e89d7cfd71fe5f3 P 2cd60cb8df9646af83deb3452da89de2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e1f390e2f6b43ef846a9189c50427e8" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 37633 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 15:01:40.146003 15865 consensus_queue.cc:1035] T b7eb5458d9f3484d9e89d7cfd71fe5f3 P 2cd60cb8df9646af83deb3452da89de2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e03dc9443b94cd09a97dc52085ffead" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 36423 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 15:01:40.147871 15886 mvcc.cc:205] Tried to move back new op lower bound from 7332559667794288640 to 7332559666928332800. Current Snapshot: MvccSnapshot[applied={T|T < 7332559667794288640}]
I20260923 15:01:40.155373 15852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:40.152366 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.129:54427 (local address 127.13.75.131:43295)
0923 15:01:40.152412 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0923 15:01:40.152414 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:01:40.152517 (+ 103us) server_negotiation.cc:407] Connection header received
0923 15:01:40.152591 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:01:40.152591 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:01:40.152600 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:01:40.152643 (+ 43us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 15:01:40.152787 (+ 144us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:40.153247 (+ 460us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:40.154517 (+ 1270us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:40.154710 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:40.154766 (+ 56us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:01:40.155071 (+ 305us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 15:01:40.155112 (+ 41us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 15:01:40.155145 (+ 33us) server_negotiation.cc:1035] Waiting for connection context
0923 15:01:40.155261 (+ 116us) server_negotiation.cc:299] Negotiation successful
0923 15:01:40.155289 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260923 15:01:40.155637 15898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:40.152289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.131:43295 (local address 127.13.75.129:54427)
0923 15:01:40.152449 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0923 15:01:40.152462 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0923 15:01:40.152529 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:01:40.152650 (+ 121us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:01:40.152651 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:01:40.152652 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0923 15:01:40.152721 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:01:40.152722 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:01:40.153321 (+ 599us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:01:40.153322 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:01:40.154429 (+ 1107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:01:40.154432 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:01:40.154791 (+ 359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:01:40.154792 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:01:40.154912 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:01:40.154933 (+ 21us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0923 15:01:40.155191 (+ 258us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0923 15:01:40.155193 (+ 2us) client_negotiation.cc:769] Sending connection context
0923 15:01:40.155476 (+ 283us) client_negotiation.cc:240] Negotiation successful
0923 15:01:40.155528 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":81,"threads_started":1}
I20260923 15:01:40.156433 15883 mvcc.cc:205] Tried to move back new op lower bound from 7332559667794288640 to 7332559666928332800. Current Snapshot: MvccSnapshot[applied={T|T < 7332559667794288640}]
I20260923 15:01:40.164132 15349 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45998:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 15:01:40.164258 15349 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45998:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 15:01:40.164978 15349 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 413f7cf8ddb348c7a6a08580e4599d5b: Sending DeleteTablet for 3 replicas of tablet b7eb5458d9f3484d9e89d7cfd71fe5f3
I20260923 15:01:40.165561 15766 tablet_service.cc:1514] Processing DeleteTablet for tablet b7eb5458d9f3484d9e89d7cfd71fe5f3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 15:01:40 UTC) from {username='kudu', principal='kudu/127.13.75.190@KRBTEST.COM'} at 127.0.0.1:55714
I20260923 15:01:40.165561 15626 tablet_service.cc:1514] Processing DeleteTablet for tablet b7eb5458d9f3484d9e89d7cfd71fe5f3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 15:01:40 UTC) from {username='kudu', principal='kudu/127.13.75.190@KRBTEST.COM'} at 127.0.0.1:51474
I20260923 15:01:40.165789 15471 tablet_service.cc:1514] Processing DeleteTablet for tablet b7eb5458d9f3484d9e89d7cfd71fe5f3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 15:01:40 UTC) from {username='kudu', principal='kudu/127.13.75.190@KRBTEST.COM'} at 127.0.0.1:40180
I20260923 15:01:40.166023 15904 tablet_replica.cc:330] T b7eb5458d9f3484d9e89d7cfd71fe5f3 P 6e03dc9443b94cd09a97dc52085ffead: stopping tablet replica
I20260923 15:01:40.166007 15349 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45998:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 15:01:40.166174 15349 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 15:01:40.166384 15904 raft_consensus.cc:2229] T b7eb5458d9f3484d9e89d7cfd71fe5f3 P 6e03dc9443b94cd09a97dc52085ffead [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 15:01:40.166528 15904 raft_consensus.cc:2258] T b7eb5458d9f3484d9e89d7cfd71fe5f3 P 6e03dc9443b94cd09a97dc52085ffead [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 15:01:40.167117 15904 ts_tablet_manager.cc:1905] T b7eb5458d9f3484d9e89d7cfd71fe5f3 P 6e03dc9443b94cd09a97dc52085ffead: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 15:01:40.167979 15766 tablet_service.cc:1467] Processing CreateTablet for tablet 2f9c0445d34e455dae1812c1d0e45011 (DEFAULT_TABLE table=test-table [id=9535f76d8e0f4d0aa8f52f12b5b5155f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 15:01:40.168123 15766 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2f9c0445d34e455dae1812c1d0e45011. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:01:40.169512 15904 ts_tablet_manager.cc:1918] T b7eb5458d9f3484d9e89d7cfd71fe5f3 P 6e03dc9443b94cd09a97dc52085ffead: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260923 15:01:40.169692 15904 log.cc:1196] T b7eb5458d9f3484d9e89d7cfd71fe5f3 P 6e03dc9443b94cd09a97dc52085ffead: Deleting WAL directory at /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175691491353-13614-0/minicluster-data/ts-0/wal/wals/b7eb5458d9f3484d9e89d7cfd71fe5f3
I20260923 15:01:40.169697 15855 tablet_bootstrap.cc:492] T 2f9c0445d34e455dae1812c1d0e45011 P 2cd60cb8df9646af83deb3452da89de2: Bootstrap starting.
I20260923 15:01:40.170197 15904 ts_tablet_manager.cc:1939] T b7eb5458d9f3484d9e89d7cfd71fe5f3 P 6e03dc9443b94cd09a97dc52085ffead: Deleting consensus metadata
I20260923 15:01:40.170231 15855 tablet_bootstrap.cc:654] T 2f9c0445d34e455dae1812c1d0e45011 P 2cd60cb8df9646af83deb3452da89de2: Neither blocks nor log segments found. Creating new log.
I20260923 15:01:40.170778 15336 catalog_manager.cc:4841] TS 6e03dc9443b94cd09a97dc52085ffead (127.13.75.129:36423): tablet b7eb5458d9f3484d9e89d7cfd71fe5f3 (table test-table [id=709481ca616f40c1b2748e27231c4598]) successfully deleted
I20260923 15:01:40.170979 15626 tablet_service.cc:1467] Processing CreateTablet for tablet 2f9c0445d34e455dae1812c1d0e45011 (DEFAULT_TABLE table=test-table [id=9535f76d8e0f4d0aa8f52f12b5b5155f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 15:01:40.171123 15626 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2f9c0445d34e455dae1812c1d0e45011. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:01:40.172072 15856 tablet_bootstrap.cc:492] T 2f9c0445d34e455dae1812c1d0e45011 P 3e1f390e2f6b43ef846a9189c50427e8: Bootstrap starting.
I20260923 15:01:40.172283 15471 tablet_service.cc:1467] Processing CreateTablet for tablet 2f9c0445d34e455dae1812c1d0e45011 (DEFAULT_TABLE table=test-table [id=9535f76d8e0f4d0aa8f52f12b5b5155f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 15:01:40.172403 15471 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2f9c0445d34e455dae1812c1d0e45011. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:01:40.172485 15856 tablet_bootstrap.cc:654] T 2f9c0445d34e455dae1812c1d0e45011 P 3e1f390e2f6b43ef846a9189c50427e8: Neither blocks nor log segments found. Creating new log.
I20260923 15:01:40.173470 15857 tablet_bootstrap.cc:492] T 2f9c0445d34e455dae1812c1d0e45011 P 6e03dc9443b94cd09a97dc52085ffead: Bootstrap starting.
I20260923 15:01:40.173888 15857 tablet_bootstrap.cc:654] T 2f9c0445d34e455dae1812c1d0e45011 P 6e03dc9443b94cd09a97dc52085ffead: Neither blocks nor log segments found. Creating new log.
I20260923 15:01:40.173899 15902 tablet_replica.cc:330] T b7eb5458d9f3484d9e89d7cfd71fe5f3 P 3e1f390e2f6b43ef846a9189c50427e8: stopping tablet replica
I20260923 15:01:40.174095 15902 raft_consensus.cc:2229] T b7eb5458d9f3484d9e89d7cfd71fe5f3 P 3e1f390e2f6b43ef846a9189c50427e8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 15:01:40.174161 15902 raft_consensus.cc:2258] T b7eb5458d9f3484d9e89d7cfd71fe5f3 P 3e1f390e2f6b43ef846a9189c50427e8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 15:01:40.174602 15857 tablet_bootstrap.cc:492] T 2f9c0445d34e455dae1812c1d0e45011 P 6e03dc9443b94cd09a97dc52085ffead: No bootstrap required, opened a new log
I20260923 15:01:40.174649 15857 ts_tablet_manager.cc:1397] T 2f9c0445d34e455dae1812c1d0e45011 P 6e03dc9443b94cd09a97dc52085ffead: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260923 15:01:40.174714 15902 ts_tablet_manager.cc:1905] T b7eb5458d9f3484d9e89d7cfd71fe5f3 P 3e1f390e2f6b43ef846a9189c50427e8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 15:01:40.174790 15857 raft_consensus.cc:348] T 2f9c0445d34e455dae1812c1d0e45011 P 6e03dc9443b94cd09a97dc52085ffead [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e03dc9443b94cd09a97dc52085ffead" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 36423 } } peers { permanent_uuid: "3e1f390e2f6b43ef846a9189c50427e8" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 37633 } } peers { permanent_uuid: "2cd60cb8df9646af83deb3452da89de2" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 43295 } }
I20260923 15:01:40.175007 15857 raft_consensus.cc:374] T 2f9c0445d34e455dae1812c1d0e45011 P 6e03dc9443b94cd09a97dc52085ffead [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:01:40.175038 15857 raft_consensus.cc:729] T 2f9c0445d34e455dae1812c1d0e45011 P 6e03dc9443b94cd09a97dc52085ffead [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e03dc9443b94cd09a97dc52085ffead, State: Initialized, Role: FOLLOWER
I20260923 15:01:40.175081 15857 consensus_queue.cc:260] T 2f9c0445d34e455dae1812c1d0e45011 P 6e03dc9443b94cd09a97dc52085ffead [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e03dc9443b94cd09a97dc52085ffead" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 36423 } } peers { permanent_uuid: "3e1f390e2f6b43ef846a9189c50427e8" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 37633 } } peers { permanent_uuid: "2cd60cb8df9646af83deb3452da89de2" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 43295 } }
I20260923 15:01:40.175194 15857 ts_tablet_manager.cc:1428] T 2f9c0445d34e455dae1812c1d0e45011 P 6e03dc9443b94cd09a97dc52085ffead: Time spent starting tablet: real 0.001s user 0.000s sys 0.000s
I20260923 15:01:40.175303 15855 tablet_bootstrap.cc:492] T 2f9c0445d34e455dae1812c1d0e45011 P 2cd60cb8df9646af83deb3452da89de2: No bootstrap required, opened a new log
I20260923 15:01:40.175354 15855 ts_tablet_manager.cc:1397] T 2f9c0445d34e455dae1812c1d0e45011 P 2cd60cb8df9646af83deb3452da89de2: Time spent bootstrapping tablet: real 0.006s user 0.001s sys 0.000s
I20260923 15:01:40.175510 15855 raft_consensus.cc:348] T 2f9c0445d34e455dae1812c1d0e45011 P 2cd60cb8df9646af83deb3452da89de2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e03dc9443b94cd09a97dc52085ffead" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 36423 } } peers { permanent_uuid: "3e1f390e2f6b43ef846a9189c50427e8" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 37633 } } peers { permanent_uuid: "2cd60cb8df9646af83deb3452da89de2" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 43295 } }
I20260923 15:01:40.175582 15855 raft_consensus.cc:374] T 2f9c0445d34e455dae1812c1d0e45011 P 2cd60cb8df9646af83deb3452da89de2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:01:40.175602 15855 raft_consensus.cc:729] T 2f9c0445d34e455dae1812c1d0e45011 P 2cd60cb8df9646af83deb3452da89de2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2cd60cb8df9646af83deb3452da89de2, State: Initialized, Role: FOLLOWER
I20260923 15:01:40.175643 15855 consensus_queue.cc:260] T 2f9c0445d34e455dae1812c1d0e45011 P 2cd60cb8df9646af83deb3452da89de2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e03dc9443b94cd09a97dc52085ffead" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 36423 } } peers { permanent_uuid: "3e1f390e2f6b43ef846a9189c50427e8" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 37633 } } peers { permanent_uuid: "2cd60cb8df9646af83deb3452da89de2" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 43295 } }
I20260923 15:01:40.175740 15855 ts_tablet_manager.cc:1428] T 2f9c0445d34e455dae1812c1d0e45011 P 2cd60cb8df9646af83deb3452da89de2: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260923 15:01:40.176831 15903 tablet_replica.cc:330] T b7eb5458d9f3484d9e89d7cfd71fe5f3 P 2cd60cb8df9646af83deb3452da89de2: stopping tablet replica
I20260923 15:01:40.176999 15903 raft_consensus.cc:2229] T b7eb5458d9f3484d9e89d7cfd71fe5f3 P 2cd60cb8df9646af83deb3452da89de2 [term 1 LEADER]: Raft consensus shutting down.
I20260923 15:01:40.177268 15903 raft_consensus.cc:2258] T b7eb5458d9f3484d9e89d7cfd71fe5f3 P 2cd60cb8df9646af83deb3452da89de2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 15:01:40.177469 15902 ts_tablet_manager.cc:1918] T b7eb5458d9f3484d9e89d7cfd71fe5f3 P 3e1f390e2f6b43ef846a9189c50427e8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260923 15:01:40.177546 15902 log.cc:1196] T b7eb5458d9f3484d9e89d7cfd71fe5f3 P 3e1f390e2f6b43ef846a9189c50427e8: Deleting WAL directory at /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175691491353-13614-0/minicluster-data/ts-1/wal/wals/b7eb5458d9f3484d9e89d7cfd71fe5f3
I20260923 15:01:40.177958 15902 ts_tablet_manager.cc:1939] T b7eb5458d9f3484d9e89d7cfd71fe5f3 P 3e1f390e2f6b43ef846a9189c50427e8: Deleting consensus metadata
I20260923 15:01:40.178000 15903 ts_tablet_manager.cc:1905] T b7eb5458d9f3484d9e89d7cfd71fe5f3 P 2cd60cb8df9646af83deb3452da89de2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 15:01:40.178426 15336 catalog_manager.cc:4841] TS 3e1f390e2f6b43ef846a9189c50427e8 (127.13.75.130:37633): tablet b7eb5458d9f3484d9e89d7cfd71fe5f3 (table test-table [id=709481ca616f40c1b2748e27231c4598]) successfully deleted
I20260923 15:01:40.179816 15903 ts_tablet_manager.cc:1918] T b7eb5458d9f3484d9e89d7cfd71fe5f3 P 2cd60cb8df9646af83deb3452da89de2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260923 15:01:40.179896 15903 log.cc:1196] T b7eb5458d9f3484d9e89d7cfd71fe5f3 P 2cd60cb8df9646af83deb3452da89de2: Deleting WAL directory at /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175691491353-13614-0/minicluster-data/ts-2/wal/wals/b7eb5458d9f3484d9e89d7cfd71fe5f3
I20260923 15:01:40.180158 15903 ts_tablet_manager.cc:1939] T b7eb5458d9f3484d9e89d7cfd71fe5f3 P 2cd60cb8df9646af83deb3452da89de2: Deleting consensus metadata
I20260923 15:01:40.180641 15333 catalog_manager.cc:4841] TS 2cd60cb8df9646af83deb3452da89de2 (127.13.75.131:43295): tablet b7eb5458d9f3484d9e89d7cfd71fe5f3 (table test-table [id=709481ca616f40c1b2748e27231c4598]) successfully deleted
I20260923 15:01:40.182977 15856 tablet_bootstrap.cc:492] T 2f9c0445d34e455dae1812c1d0e45011 P 3e1f390e2f6b43ef846a9189c50427e8: No bootstrap required, opened a new log
I20260923 15:01:40.183044 15856 ts_tablet_manager.cc:1397] T 2f9c0445d34e455dae1812c1d0e45011 P 3e1f390e2f6b43ef846a9189c50427e8: Time spent bootstrapping tablet: real 0.011s user 0.001s sys 0.000s
I20260923 15:01:40.183208 15856 raft_consensus.cc:348] T 2f9c0445d34e455dae1812c1d0e45011 P 3e1f390e2f6b43ef846a9189c50427e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e03dc9443b94cd09a97dc52085ffead" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 36423 } } peers { permanent_uuid: "3e1f390e2f6b43ef846a9189c50427e8" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 37633 } } peers { permanent_uuid: "2cd60cb8df9646af83deb3452da89de2" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 43295 } }
I20260923 15:01:40.183291 15856 raft_consensus.cc:374] T 2f9c0445d34e455dae1812c1d0e45011 P 3e1f390e2f6b43ef846a9189c50427e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:01:40.183311 15856 raft_consensus.cc:729] T 2f9c0445d34e455dae1812c1d0e45011 P 3e1f390e2f6b43ef846a9189c50427e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e1f390e2f6b43ef846a9189c50427e8, State: Initialized, Role: FOLLOWER
I20260923 15:01:40.183359 15856 consensus_queue.cc:260] T 2f9c0445d34e455dae1812c1d0e45011 P 3e1f390e2f6b43ef846a9189c50427e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e03dc9443b94cd09a97dc52085ffead" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 36423 } } peers { permanent_uuid: "3e1f390e2f6b43ef846a9189c50427e8" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 37633 } } peers { permanent_uuid: "2cd60cb8df9646af83deb3452da89de2" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 43295 } }
I20260923 15:01:40.183445 15856 ts_tablet_manager.cc:1428] T 2f9c0445d34e455dae1812c1d0e45011 P 3e1f390e2f6b43ef846a9189c50427e8: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260923 15:01:40.260363 15900 raft_consensus.cc:482] T 2f9c0445d34e455dae1812c1d0e45011 P 2cd60cb8df9646af83deb3452da89de2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 15:01:40.260483 15900 raft_consensus.cc:504] T 2f9c0445d34e455dae1812c1d0e45011 P 2cd60cb8df9646af83deb3452da89de2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e03dc9443b94cd09a97dc52085ffead" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 36423 } } peers { permanent_uuid: "3e1f390e2f6b43ef846a9189c50427e8" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 37633 } } peers { permanent_uuid: "2cd60cb8df9646af83deb3452da89de2" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 43295 } }
I20260923 15:01:40.260641 15900 leader_election.cc:288] T 2f9c0445d34e455dae1812c1d0e45011 P 2cd60cb8df9646af83deb3452da89de2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e03dc9443b94cd09a97dc52085ffead (127.13.75.129:36423), 3e1f390e2f6b43ef846a9189c50427e8 (127.13.75.130:37633)
I20260923 15:01:40.260921 15497 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2f9c0445d34e455dae1812c1d0e45011" candidate_uuid: "2cd60cb8df9646af83deb3452da89de2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e03dc9443b94cd09a97dc52085ffead" is_pre_election: true
I20260923 15:01:40.261010 15497 raft_consensus.cc:2454] T 2f9c0445d34e455dae1812c1d0e45011 P 6e03dc9443b94cd09a97dc52085ffead [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2cd60cb8df9646af83deb3452da89de2 in term 0.
I20260923 15:01:40.260998 15646 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2f9c0445d34e455dae1812c1d0e45011" candidate_uuid: "2cd60cb8df9646af83deb3452da89de2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e1f390e2f6b43ef846a9189c50427e8" is_pre_election: true
I20260923 15:01:40.261121 15646 raft_consensus.cc:2454] T 2f9c0445d34e455dae1812c1d0e45011 P 3e1f390e2f6b43ef846a9189c50427e8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2cd60cb8df9646af83deb3452da89de2 in term 0.
I20260923 15:01:40.261193 15720 leader_election.cc:302] T 2f9c0445d34e455dae1812c1d0e45011 P 2cd60cb8df9646af83deb3452da89de2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2cd60cb8df9646af83deb3452da89de2, 6e03dc9443b94cd09a97dc52085ffead; no voters:
I20260923 15:01:40.261329 15900 raft_consensus.cc:2781] T 2f9c0445d34e455dae1812c1d0e45011 P 2cd60cb8df9646af83deb3452da89de2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 15:01:40.261363 15900 raft_consensus.cc:482] T 2f9c0445d34e455dae1812c1d0e45011 P 2cd60cb8df9646af83deb3452da89de2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 15:01:40.261384 15900 raft_consensus.cc:3037] T 2f9c0445d34e455dae1812c1d0e45011 P 2cd60cb8df9646af83deb3452da89de2 [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:01:40.262347 15900 raft_consensus.cc:504] T 2f9c0445d34e455dae1812c1d0e45011 P 2cd60cb8df9646af83deb3452da89de2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e03dc9443b94cd09a97dc52085ffead" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 36423 } } peers { permanent_uuid: "3e1f390e2f6b43ef846a9189c50427e8" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 37633 } } peers { permanent_uuid: "2cd60cb8df9646af83deb3452da89de2" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 43295 } }
I20260923 15:01:40.262498 15900 leader_election.cc:288] T 2f9c0445d34e455dae1812c1d0e45011 P 2cd60cb8df9646af83deb3452da89de2 [CANDIDATE]: Term 1 election: Requested vote from peers 6e03dc9443b94cd09a97dc52085ffead (127.13.75.129:36423), 3e1f390e2f6b43ef846a9189c50427e8 (127.13.75.130:37633)
I20260923 15:01:40.262732 15497 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2f9c0445d34e455dae1812c1d0e45011" candidate_uuid: "2cd60cb8df9646af83deb3452da89de2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e03dc9443b94cd09a97dc52085ffead"
I20260923 15:01:40.262751 15646 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2f9c0445d34e455dae1812c1d0e45011" candidate_uuid: "2cd60cb8df9646af83deb3452da89de2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e1f390e2f6b43ef846a9189c50427e8"
I20260923 15:01:40.262805 15497 raft_consensus.cc:3037] T 2f9c0445d34e455dae1812c1d0e45011 P 6e03dc9443b94cd09a97dc52085ffead [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:01:40.262835 15646 raft_consensus.cc:3037] T 2f9c0445d34e455dae1812c1d0e45011 P 3e1f390e2f6b43ef846a9189c50427e8 [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:01:40.263605 15497 raft_consensus.cc:2454] T 2f9c0445d34e455dae1812c1d0e45011 P 6e03dc9443b94cd09a97dc52085ffead [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2cd60cb8df9646af83deb3452da89de2 in term 1.
I20260923 15:01:40.263710 15646 raft_consensus.cc:2454] T 2f9c0445d34e455dae1812c1d0e45011 P 3e1f390e2f6b43ef846a9189c50427e8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2cd60cb8df9646af83deb3452da89de2 in term 1.
I20260923 15:01:40.263792 15720 leader_election.cc:302] T 2f9c0445d34e455dae1812c1d0e45011 P 2cd60cb8df9646af83deb3452da89de2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2cd60cb8df9646af83deb3452da89de2, 6e03dc9443b94cd09a97dc52085ffead; no voters:
I20260923 15:01:40.263890 15900 raft_consensus.cc:2781] T 2f9c0445d34e455dae1812c1d0e45011 P 2cd60cb8df9646af83deb3452da89de2 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 15:01:40.263926 15900 raft_consensus.cc:686] T 2f9c0445d34e455dae1812c1d0e45011 P 2cd60cb8df9646af83deb3452da89de2 [term 1 LEADER]: Becoming Leader. State: Replica: 2cd60cb8df9646af83deb3452da89de2, State: Running, Role: LEADER
I20260923 15:01:40.263968 15900 consensus_queue.cc:237] T 2f9c0445d34e455dae1812c1d0e45011 P 2cd60cb8df9646af83deb3452da89de2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e03dc9443b94cd09a97dc52085ffead" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 36423 } } peers { permanent_uuid: "3e1f390e2f6b43ef846a9189c50427e8" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 37633 } } peers { permanent_uuid: "2cd60cb8df9646af83deb3452da89de2" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 43295 } }
I20260923 15:01:40.264664 15349 catalog_manager.cc:5495] T 2f9c0445d34e455dae1812c1d0e45011 P 2cd60cb8df9646af83deb3452da89de2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2cd60cb8df9646af83deb3452da89de2 (127.13.75.131). New cstate: current_term: 1 leader_uuid: "2cd60cb8df9646af83deb3452da89de2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e03dc9443b94cd09a97dc52085ffead" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 36423 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e1f390e2f6b43ef846a9189c50427e8" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 37633 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2cd60cb8df9646af83deb3452da89de2" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 43295 } health_report { overall_health: HEALTHY } } }
I20260923 15:01:40.272754 15645 raft_consensus.cc:1261] T 2f9c0445d34e455dae1812c1d0e45011 P 3e1f390e2f6b43ef846a9189c50427e8 [term 1 FOLLOWER]: Refusing update from remote peer 2cd60cb8df9646af83deb3452da89de2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 15:01:40.272754 15497 raft_consensus.cc:1261] T 2f9c0445d34e455dae1812c1d0e45011 P 6e03dc9443b94cd09a97dc52085ffead [term 1 FOLLOWER]: Refusing update from remote peer 2cd60cb8df9646af83deb3452da89de2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 15:01:40.273065 15900 consensus_queue.cc:1035] T 2f9c0445d34e455dae1812c1d0e45011 P 2cd60cb8df9646af83deb3452da89de2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e1f390e2f6b43ef846a9189c50427e8" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 37633 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 15:01:40.273188 15865 consensus_queue.cc:1035] T 2f9c0445d34e455dae1812c1d0e45011 P 2cd60cb8df9646af83deb3452da89de2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e03dc9443b94cd09a97dc52085ffead" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 36423 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260923 15:01:40.278780 15886 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260923 15:01:41.802242 15349 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45998:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 15:01:41.802356 15349 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:45998:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 15:01:41.803297 15349 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 413f7cf8ddb348c7a6a08580e4599d5b: Sending DeleteTablet for 3 replicas of tablet 2f9c0445d34e455dae1812c1d0e45011
I20260923 15:01:41.803593 15766 tablet_service.cc:1514] Processing DeleteTablet for tablet 2f9c0445d34e455dae1812c1d0e45011 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 15:01:41 UTC) from {username='kudu', principal='kudu/127.13.75.190@KRBTEST.COM'} at 127.0.0.1:55714
I20260923 15:01:41.803659 15626 tablet_service.cc:1514] Processing DeleteTablet for tablet 2f9c0445d34e455dae1812c1d0e45011 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 15:01:41 UTC) from {username='kudu', principal='kudu/127.13.75.190@KRBTEST.COM'} at 127.0.0.1:51474
I20260923 15:01:41.803838 15471 tablet_service.cc:1514] Processing DeleteTablet for tablet 2f9c0445d34e455dae1812c1d0e45011 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 15:01:41 UTC) from {username='kudu', principal='kudu/127.13.75.190@KRBTEST.COM'} at 127.0.0.1:40180
I20260923 15:01:41.804028 15934 tablet_replica.cc:330] T 2f9c0445d34e455dae1812c1d0e45011 P 6e03dc9443b94cd09a97dc52085ffead: stopping tablet replica
I20260923 15:01:41.804044 15932 tablet_replica.cc:330] T 2f9c0445d34e455dae1812c1d0e45011 P 2cd60cb8df9646af83deb3452da89de2: stopping tablet replica
I20260923 15:01:41.804055 15933 tablet_replica.cc:330] T 2f9c0445d34e455dae1812c1d0e45011 P 3e1f390e2f6b43ef846a9189c50427e8: stopping tablet replica
I20260923 15:01:41.804100 15932 raft_consensus.cc:2229] T 2f9c0445d34e455dae1812c1d0e45011 P 2cd60cb8df9646af83deb3452da89de2 [term 1 LEADER]: Raft consensus shutting down.
I20260923 15:01:41.804101 15933 raft_consensus.cc:2229] T 2f9c0445d34e455dae1812c1d0e45011 P 3e1f390e2f6b43ef846a9189c50427e8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 15:01:41.804150 15933 raft_consensus.cc:2258] T 2f9c0445d34e455dae1812c1d0e45011 P 3e1f390e2f6b43ef846a9189c50427e8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 15:01:41.804175 15932 raft_consensus.cc:2258] T 2f9c0445d34e455dae1812c1d0e45011 P 2cd60cb8df9646af83deb3452da89de2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 15:01:41.804255 15934 raft_consensus.cc:2229] T 2f9c0445d34e455dae1812c1d0e45011 P 6e03dc9443b94cd09a97dc52085ffead [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 15:01:41.804340 15934 raft_consensus.cc:2258] T 2f9c0445d34e455dae1812c1d0e45011 P 6e03dc9443b94cd09a97dc52085ffead [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 15:01:41.804415 15933 ts_tablet_manager.cc:1905] T 2f9c0445d34e455dae1812c1d0e45011 P 3e1f390e2f6b43ef846a9189c50427e8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 15:01:41.804415 15932 ts_tablet_manager.cc:1905] T 2f9c0445d34e455dae1812c1d0e45011 P 2cd60cb8df9646af83deb3452da89de2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 15:01:41.804678 15934 ts_tablet_manager.cc:1905] T 2f9c0445d34e455dae1812c1d0e45011 P 6e03dc9443b94cd09a97dc52085ffead: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 15:01:41.806109 15932 ts_tablet_manager.cc:1918] T 2f9c0445d34e455dae1812c1d0e45011 P 2cd60cb8df9646af83deb3452da89de2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260923 15:01:41.806193 15932 log.cc:1196] T 2f9c0445d34e455dae1812c1d0e45011 P 2cd60cb8df9646af83deb3452da89de2: Deleting WAL directory at /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175691491353-13614-0/minicluster-data/ts-2/wal/wals/2f9c0445d34e455dae1812c1d0e45011
I20260923 15:01:41.806481 15932 ts_tablet_manager.cc:1939] T 2f9c0445d34e455dae1812c1d0e45011 P 2cd60cb8df9646af83deb3452da89de2: Deleting consensus metadata
I20260923 15:01:41.806672 15934 ts_tablet_manager.cc:1918] T 2f9c0445d34e455dae1812c1d0e45011 P 6e03dc9443b94cd09a97dc52085ffead: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260923 15:01:41.806751 15934 log.cc:1196] T 2f9c0445d34e455dae1812c1d0e45011 P 6e03dc9443b94cd09a97dc52085ffead: Deleting WAL directory at /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175691491353-13614-0/minicluster-data/ts-0/wal/wals/2f9c0445d34e455dae1812c1d0e45011
I20260923 15:01:41.806802 15933 ts_tablet_manager.cc:1918] T 2f9c0445d34e455dae1812c1d0e45011 P 3e1f390e2f6b43ef846a9189c50427e8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260923 15:01:41.806857 15933 log.cc:1196] T 2f9c0445d34e455dae1812c1d0e45011 P 3e1f390e2f6b43ef846a9189c50427e8: Deleting WAL directory at /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175691491353-13614-0/minicluster-data/ts-1/wal/wals/2f9c0445d34e455dae1812c1d0e45011
I20260923 15:01:41.806850 15333 catalog_manager.cc:4841] TS 2cd60cb8df9646af83deb3452da89de2 (127.13.75.131:43295): tablet 2f9c0445d34e455dae1812c1d0e45011 (table test-table [id=9535f76d8e0f4d0aa8f52f12b5b5155f]) successfully deleted
I20260923 15:01:41.807008 15934 ts_tablet_manager.cc:1939] T 2f9c0445d34e455dae1812c1d0e45011 P 6e03dc9443b94cd09a97dc52085ffead: Deleting consensus metadata
I20260923 15:01:41.807133 15933 ts_tablet_manager.cc:1939] T 2f9c0445d34e455dae1812c1d0e45011 P 3e1f390e2f6b43ef846a9189c50427e8: Deleting consensus metadata
I20260923 15:01:41.807333 15336 catalog_manager.cc:4841] TS 6e03dc9443b94cd09a97dc52085ffead (127.13.75.129:36423): tablet 2f9c0445d34e455dae1812c1d0e45011 (table test-table [id=9535f76d8e0f4d0aa8f52f12b5b5155f]) successfully deleted
I20260923 15:01:41.807415 15336 catalog_manager.cc:4841] TS 3e1f390e2f6b43ef846a9189c50427e8 (127.13.75.130:37633): tablet 2f9c0445d34e455dae1812c1d0e45011 (table test-table [id=9535f76d8e0f4d0aa8f52f12b5b5155f]) successfully deleted
Sep 23 15:01:41 dist-test-slave-cnb0 krb5kdc[15304](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175701, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 23 15:01:41 dist-test-slave-cnb0 krb5kdc[15304](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175701, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for kudu/127.13.75.190@KRBTEST.COM
I20260923 15:01:41.834395 15942 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:41.820656 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46020 (local address 127.13.75.190:43703)
0923 15:01:41.820828 (+ 172us) server_negotiation.cc:206] Beginning negotiation
0923 15:01:41.820830 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:01:41.820842 (+ 12us) server_negotiation.cc:407] Connection header received
0923 15:01:41.820899 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:01:41.820899 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:01:41.820910 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:01:41.820996 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:01:41.821632 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:41.822281 (+ 649us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:41.823144 (+ 863us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:41.823344 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:41.823455 (+ 111us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:01:41.831764 (+ 8309us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:01:41.831766 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:01:41.831766 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:01:41.831776 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:01:41.833237 (+ 1461us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:01:41.833635 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:01:41.833636 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:01:41.833637 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:01:41.833693 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:01:41.833862 (+ 169us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:01:41.833862 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:01:41.833863 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:01:41.833995 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:01:41.834073 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0923 15:01:41.834190 (+ 117us) server_negotiation.cc:299] Negotiation successful
0923 15:01:41.834261 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":111,"threads_started":1}
I20260923 15:01:41.836119 15349 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46020:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 15:01:41.836275 15349 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 15:01:41.837855 15471 tablet_service.cc:1467] Processing CreateTablet for tablet 68a83f9b46f04efea7c3139289aa7aeb (DEFAULT_TABLE table=test-table [id=f94bf9522b504e75b9c08554fff2e65c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 15:01:41.837874 15626 tablet_service.cc:1467] Processing CreateTablet for tablet 68a83f9b46f04efea7c3139289aa7aeb (DEFAULT_TABLE table=test-table [id=f94bf9522b504e75b9c08554fff2e65c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 15:01:41.837898 15766 tablet_service.cc:1467] Processing CreateTablet for tablet 68a83f9b46f04efea7c3139289aa7aeb (DEFAULT_TABLE table=test-table [id=f94bf9522b504e75b9c08554fff2e65c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 15:01:41.838016 15626 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 68a83f9b46f04efea7c3139289aa7aeb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:01:41.838016 15471 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 68a83f9b46f04efea7c3139289aa7aeb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:01:41.838090 15766 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 68a83f9b46f04efea7c3139289aa7aeb. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:01:41.839336 15943 tablet_bootstrap.cc:492] T 68a83f9b46f04efea7c3139289aa7aeb P 2cd60cb8df9646af83deb3452da89de2: Bootstrap starting.
I20260923 15:01:41.839516 15944 tablet_bootstrap.cc:492] T 68a83f9b46f04efea7c3139289aa7aeb P 3e1f390e2f6b43ef846a9189c50427e8: Bootstrap starting.
I20260923 15:01:41.839915 15943 tablet_bootstrap.cc:654] T 68a83f9b46f04efea7c3139289aa7aeb P 2cd60cb8df9646af83deb3452da89de2: Neither blocks nor log segments found. Creating new log.
I20260923 15:01:41.839915 15944 tablet_bootstrap.cc:654] T 68a83f9b46f04efea7c3139289aa7aeb P 3e1f390e2f6b43ef846a9189c50427e8: Neither blocks nor log segments found. Creating new log.
I20260923 15:01:41.840232 15945 tablet_bootstrap.cc:492] T 68a83f9b46f04efea7c3139289aa7aeb P 6e03dc9443b94cd09a97dc52085ffead: Bootstrap starting.
I20260923 15:01:41.840559 15943 tablet_bootstrap.cc:492] T 68a83f9b46f04efea7c3139289aa7aeb P 2cd60cb8df9646af83deb3452da89de2: No bootstrap required, opened a new log
I20260923 15:01:41.840600 15945 tablet_bootstrap.cc:654] T 68a83f9b46f04efea7c3139289aa7aeb P 6e03dc9443b94cd09a97dc52085ffead: Neither blocks nor log segments found. Creating new log.
I20260923 15:01:41.840620 15943 ts_tablet_manager.cc:1397] T 68a83f9b46f04efea7c3139289aa7aeb P 2cd60cb8df9646af83deb3452da89de2: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260923 15:01:41.840781 15943 raft_consensus.cc:348] T 68a83f9b46f04efea7c3139289aa7aeb P 2cd60cb8df9646af83deb3452da89de2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e03dc9443b94cd09a97dc52085ffead" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 36423 } } peers { permanent_uuid: "3e1f390e2f6b43ef846a9189c50427e8" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 37633 } } peers { permanent_uuid: "2cd60cb8df9646af83deb3452da89de2" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 43295 } }
I20260923 15:01:41.840868 15943 raft_consensus.cc:374] T 68a83f9b46f04efea7c3139289aa7aeb P 2cd60cb8df9646af83deb3452da89de2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:01:41.840888 15943 raft_consensus.cc:729] T 68a83f9b46f04efea7c3139289aa7aeb P 2cd60cb8df9646af83deb3452da89de2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2cd60cb8df9646af83deb3452da89de2, State: Initialized, Role: FOLLOWER
I20260923 15:01:41.840935 15943 consensus_queue.cc:260] T 68a83f9b46f04efea7c3139289aa7aeb P 2cd60cb8df9646af83deb3452da89de2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e03dc9443b94cd09a97dc52085ffead" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 36423 } } peers { permanent_uuid: "3e1f390e2f6b43ef846a9189c50427e8" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 37633 } } peers { permanent_uuid: "2cd60cb8df9646af83deb3452da89de2" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 43295 } }
I20260923 15:01:41.841095 15943 ts_tablet_manager.cc:1428] T 68a83f9b46f04efea7c3139289aa7aeb P 2cd60cb8df9646af83deb3452da89de2: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260923 15:01:41.841307 15944 tablet_bootstrap.cc:492] T 68a83f9b46f04efea7c3139289aa7aeb P 3e1f390e2f6b43ef846a9189c50427e8: No bootstrap required, opened a new log
I20260923 15:01:41.841310 15945 tablet_bootstrap.cc:492] T 68a83f9b46f04efea7c3139289aa7aeb P 6e03dc9443b94cd09a97dc52085ffead: No bootstrap required, opened a new log
I20260923 15:01:41.841354 15944 ts_tablet_manager.cc:1397] T 68a83f9b46f04efea7c3139289aa7aeb P 3e1f390e2f6b43ef846a9189c50427e8: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 15:01:41.841354 15945 ts_tablet_manager.cc:1397] T 68a83f9b46f04efea7c3139289aa7aeb P 6e03dc9443b94cd09a97dc52085ffead: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260923 15:01:41.841496 15944 raft_consensus.cc:348] T 68a83f9b46f04efea7c3139289aa7aeb P 3e1f390e2f6b43ef846a9189c50427e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e03dc9443b94cd09a97dc52085ffead" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 36423 } } peers { permanent_uuid: "3e1f390e2f6b43ef846a9189c50427e8" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 37633 } } peers { permanent_uuid: "2cd60cb8df9646af83deb3452da89de2" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 43295 } }
I20260923 15:01:41.841568 15944 raft_consensus.cc:374] T 68a83f9b46f04efea7c3139289aa7aeb P 3e1f390e2f6b43ef846a9189c50427e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:01:41.841589 15944 raft_consensus.cc:729] T 68a83f9b46f04efea7c3139289aa7aeb P 3e1f390e2f6b43ef846a9189c50427e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e1f390e2f6b43ef846a9189c50427e8, State: Initialized, Role: FOLLOWER
I20260923 15:01:41.841605 15945 raft_consensus.cc:348] T 68a83f9b46f04efea7c3139289aa7aeb P 6e03dc9443b94cd09a97dc52085ffead [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e03dc9443b94cd09a97dc52085ffead" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 36423 } } peers { permanent_uuid: "3e1f390e2f6b43ef846a9189c50427e8" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 37633 } } peers { permanent_uuid: "2cd60cb8df9646af83deb3452da89de2" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 43295 } }
I20260923 15:01:41.841638 15944 consensus_queue.cc:260] T 68a83f9b46f04efea7c3139289aa7aeb P 3e1f390e2f6b43ef846a9189c50427e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e03dc9443b94cd09a97dc52085ffead" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 36423 } } peers { permanent_uuid: "3e1f390e2f6b43ef846a9189c50427e8" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 37633 } } peers { permanent_uuid: "2cd60cb8df9646af83deb3452da89de2" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 43295 } }
I20260923 15:01:41.841684 15945 raft_consensus.cc:374] T 68a83f9b46f04efea7c3139289aa7aeb P 6e03dc9443b94cd09a97dc52085ffead [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:01:41.841717 15945 raft_consensus.cc:729] T 68a83f9b46f04efea7c3139289aa7aeb P 6e03dc9443b94cd09a97dc52085ffead [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e03dc9443b94cd09a97dc52085ffead, State: Initialized, Role: FOLLOWER
I20260923 15:01:41.841804 15944 ts_tablet_manager.cc:1428] T 68a83f9b46f04efea7c3139289aa7aeb P 3e1f390e2f6b43ef846a9189c50427e8: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260923 15:01:41.841785 15945 consensus_queue.cc:260] T 68a83f9b46f04efea7c3139289aa7aeb P 6e03dc9443b94cd09a97dc52085ffead [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e03dc9443b94cd09a97dc52085ffead" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 36423 } } peers { permanent_uuid: "3e1f390e2f6b43ef846a9189c50427e8" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 37633 } } peers { permanent_uuid: "2cd60cb8df9646af83deb3452da89de2" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 43295 } }
I20260923 15:01:41.841938 15945 ts_tablet_manager.cc:1428] T 68a83f9b46f04efea7c3139289aa7aeb P 6e03dc9443b94cd09a97dc52085ffead: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260923 15:01:41.894474 15861 raft_consensus.cc:482] T 68a83f9b46f04efea7c3139289aa7aeb P 2cd60cb8df9646af83deb3452da89de2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 15:01:41.894574 15861 raft_consensus.cc:504] T 68a83f9b46f04efea7c3139289aa7aeb P 2cd60cb8df9646af83deb3452da89de2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e03dc9443b94cd09a97dc52085ffead" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 36423 } } peers { permanent_uuid: "3e1f390e2f6b43ef846a9189c50427e8" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 37633 } } peers { permanent_uuid: "2cd60cb8df9646af83deb3452da89de2" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 43295 } }
I20260923 15:01:41.894724 15861 leader_election.cc:288] T 68a83f9b46f04efea7c3139289aa7aeb P 2cd60cb8df9646af83deb3452da89de2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e03dc9443b94cd09a97dc52085ffead (127.13.75.129:36423), 3e1f390e2f6b43ef846a9189c50427e8 (127.13.75.130:37633)
I20260923 15:01:41.894987 15497 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "68a83f9b46f04efea7c3139289aa7aeb" candidate_uuid: "2cd60cb8df9646af83deb3452da89de2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e03dc9443b94cd09a97dc52085ffead" is_pre_election: true
I20260923 15:01:41.895083 15497 raft_consensus.cc:2454] T 68a83f9b46f04efea7c3139289aa7aeb P 6e03dc9443b94cd09a97dc52085ffead [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2cd60cb8df9646af83deb3452da89de2 in term 0.
I20260923 15:01:41.895054 15646 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "68a83f9b46f04efea7c3139289aa7aeb" candidate_uuid: "2cd60cb8df9646af83deb3452da89de2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e1f390e2f6b43ef846a9189c50427e8" is_pre_election: true
I20260923 15:01:41.895179 15646 raft_consensus.cc:2454] T 68a83f9b46f04efea7c3139289aa7aeb P 3e1f390e2f6b43ef846a9189c50427e8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2cd60cb8df9646af83deb3452da89de2 in term 0.
I20260923 15:01:41.895319 15720 leader_election.cc:302] T 68a83f9b46f04efea7c3139289aa7aeb P 2cd60cb8df9646af83deb3452da89de2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2cd60cb8df9646af83deb3452da89de2, 6e03dc9443b94cd09a97dc52085ffead; no voters:
I20260923 15:01:41.895435 15861 raft_consensus.cc:2781] T 68a83f9b46f04efea7c3139289aa7aeb P 2cd60cb8df9646af83deb3452da89de2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 15:01:41.895476 15861 raft_consensus.cc:482] T 68a83f9b46f04efea7c3139289aa7aeb P 2cd60cb8df9646af83deb3452da89de2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 15:01:41.895500 15861 raft_consensus.cc:3037] T 68a83f9b46f04efea7c3139289aa7aeb P 2cd60cb8df9646af83deb3452da89de2 [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:01:41.896176 15861 raft_consensus.cc:504] T 68a83f9b46f04efea7c3139289aa7aeb P 2cd60cb8df9646af83deb3452da89de2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e03dc9443b94cd09a97dc52085ffead" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 36423 } } peers { permanent_uuid: "3e1f390e2f6b43ef846a9189c50427e8" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 37633 } } peers { permanent_uuid: "2cd60cb8df9646af83deb3452da89de2" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 43295 } }
I20260923 15:01:41.896302 15861 leader_election.cc:288] T 68a83f9b46f04efea7c3139289aa7aeb P 2cd60cb8df9646af83deb3452da89de2 [CANDIDATE]: Term 1 election: Requested vote from peers 6e03dc9443b94cd09a97dc52085ffead (127.13.75.129:36423), 3e1f390e2f6b43ef846a9189c50427e8 (127.13.75.130:37633)
I20260923 15:01:41.896445 15497 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "68a83f9b46f04efea7c3139289aa7aeb" candidate_uuid: "2cd60cb8df9646af83deb3452da89de2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e03dc9443b94cd09a97dc52085ffead"
I20260923 15:01:41.896522 15497 raft_consensus.cc:3037] T 68a83f9b46f04efea7c3139289aa7aeb P 6e03dc9443b94cd09a97dc52085ffead [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:01:41.896484 15646 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "68a83f9b46f04efea7c3139289aa7aeb" candidate_uuid: "2cd60cb8df9646af83deb3452da89de2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e1f390e2f6b43ef846a9189c50427e8"
I20260923 15:01:41.896567 15646 raft_consensus.cc:3037] T 68a83f9b46f04efea7c3139289aa7aeb P 3e1f390e2f6b43ef846a9189c50427e8 [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:01:41.897250 15497 raft_consensus.cc:2454] T 68a83f9b46f04efea7c3139289aa7aeb P 6e03dc9443b94cd09a97dc52085ffead [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2cd60cb8df9646af83deb3452da89de2 in term 1.
I20260923 15:01:41.897333 15646 raft_consensus.cc:2454] T 68a83f9b46f04efea7c3139289aa7aeb P 3e1f390e2f6b43ef846a9189c50427e8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2cd60cb8df9646af83deb3452da89de2 in term 1.
I20260923 15:01:41.897425 15720 leader_election.cc:302] T 68a83f9b46f04efea7c3139289aa7aeb P 2cd60cb8df9646af83deb3452da89de2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2cd60cb8df9646af83deb3452da89de2, 6e03dc9443b94cd09a97dc52085ffead; no voters:
I20260923 15:01:41.897526 15861 raft_consensus.cc:2781] T 68a83f9b46f04efea7c3139289aa7aeb P 2cd60cb8df9646af83deb3452da89de2 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 15:01:41.897596 15861 raft_consensus.cc:686] T 68a83f9b46f04efea7c3139289aa7aeb P 2cd60cb8df9646af83deb3452da89de2 [term 1 LEADER]: Becoming Leader. State: Replica: 2cd60cb8df9646af83deb3452da89de2, State: Running, Role: LEADER
I20260923 15:01:41.897655 15861 consensus_queue.cc:237] T 68a83f9b46f04efea7c3139289aa7aeb P 2cd60cb8df9646af83deb3452da89de2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e03dc9443b94cd09a97dc52085ffead" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 36423 } } peers { permanent_uuid: "3e1f390e2f6b43ef846a9189c50427e8" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 37633 } } peers { permanent_uuid: "2cd60cb8df9646af83deb3452da89de2" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 43295 } }
I20260923 15:01:41.898270 15348 catalog_manager.cc:5495] T 68a83f9b46f04efea7c3139289aa7aeb P 2cd60cb8df9646af83deb3452da89de2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2cd60cb8df9646af83deb3452da89de2 (127.13.75.131). New cstate: current_term: 1 leader_uuid: "2cd60cb8df9646af83deb3452da89de2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e03dc9443b94cd09a97dc52085ffead" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 36423 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e1f390e2f6b43ef846a9189c50427e8" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 37633 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2cd60cb8df9646af83deb3452da89de2" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 43295 } health_report { overall_health: HEALTHY } } }
I20260923 15:01:41.918334 15952 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:41.915855 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55742 (local address 127.13.75.131:43295)
0923 15:01:41.916010 (+ 155us) server_negotiation.cc:206] Beginning negotiation
0923 15:01:41.916012 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:01:41.916021 (+ 9us) server_negotiation.cc:407] Connection header received
0923 15:01:41.916045 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:01:41.916045 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:01:41.916055 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:01:41.916129 (+ 74us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 15:01:41.916279 (+ 150us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:41.916808 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:41.917615 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:41.917840 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:41.917916 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:01:41.918074 (+ 158us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 15:01:41.918110 (+ 36us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 15:01:41.918155 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0923 15:01:41.918209 (+ 54us) server_negotiation.cc:299] Negotiation successful
0923 15:01:41.918232 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":79,"threads_started":1}
I20260923 15:01:41.920650 15646 raft_consensus.cc:1261] T 68a83f9b46f04efea7c3139289aa7aeb P 3e1f390e2f6b43ef846a9189c50427e8 [term 1 FOLLOWER]: Refusing update from remote peer 2cd60cb8df9646af83deb3452da89de2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 15:01:41.920652 15497 raft_consensus.cc:1261] T 68a83f9b46f04efea7c3139289aa7aeb P 6e03dc9443b94cd09a97dc52085ffead [term 1 FOLLOWER]: Refusing update from remote peer 2cd60cb8df9646af83deb3452da89de2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 15:01:41.920926 15923 consensus_queue.cc:1035] T 68a83f9b46f04efea7c3139289aa7aeb P 2cd60cb8df9646af83deb3452da89de2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e03dc9443b94cd09a97dc52085ffead" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 36423 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 15:01:41.921053 15861 consensus_queue.cc:1035] T 68a83f9b46f04efea7c3139289aa7aeb P 2cd60cb8df9646af83deb3452da89de2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e1f390e2f6b43ef846a9189c50427e8" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 37633 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 15:01:41.930792 15349 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46020:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 15:01:41.930935 15349 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46020:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 15:01:41.931563 15349 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 413f7cf8ddb348c7a6a08580e4599d5b: Sending DeleteTablet for 3 replicas of tablet 68a83f9b46f04efea7c3139289aa7aeb
I20260923 15:01:41.931794 15471 tablet_service.cc:1514] Processing DeleteTablet for tablet 68a83f9b46f04efea7c3139289aa7aeb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 15:01:41 UTC) from {username='kudu', principal='kudu/127.13.75.190@KRBTEST.COM'} at 127.0.0.1:40180
I20260923 15:01:41.931855 15766 tablet_service.cc:1514] Processing DeleteTablet for tablet 68a83f9b46f04efea7c3139289aa7aeb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 15:01:41 UTC) from {username='kudu', principal='kudu/127.13.75.190@KRBTEST.COM'} at 127.0.0.1:55714
I20260923 15:01:41.931900 15934 tablet_replica.cc:330] T 68a83f9b46f04efea7c3139289aa7aeb P 6e03dc9443b94cd09a97dc52085ffead: stopping tablet replica
I20260923 15:01:41.931929 15626 tablet_service.cc:1514] Processing DeleteTablet for tablet 68a83f9b46f04efea7c3139289aa7aeb with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 15:01:41 UTC) from {username='kudu', principal='kudu/127.13.75.190@KRBTEST.COM'} at 127.0.0.1:51474
I20260923 15:01:41.931962 15934 raft_consensus.cc:2229] T 68a83f9b46f04efea7c3139289aa7aeb P 6e03dc9443b94cd09a97dc52085ffead [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 15:01:41.932006 15934 raft_consensus.cc:2258] T 68a83f9b46f04efea7c3139289aa7aeb P 6e03dc9443b94cd09a97dc52085ffead [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 15:01:41.932018 15932 tablet_replica.cc:330] T 68a83f9b46f04efea7c3139289aa7aeb P 2cd60cb8df9646af83deb3452da89de2: stopping tablet replica
I20260923 15:01:41.932068 15932 raft_consensus.cc:2229] T 68a83f9b46f04efea7c3139289aa7aeb P 2cd60cb8df9646af83deb3452da89de2 [term 1 LEADER]: Raft consensus shutting down.
I20260923 15:01:41.932114 15932 raft_consensus.cc:2258] T 68a83f9b46f04efea7c3139289aa7aeb P 2cd60cb8df9646af83deb3452da89de2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 15:01:41.932178 15933 tablet_replica.cc:330] T 68a83f9b46f04efea7c3139289aa7aeb P 3e1f390e2f6b43ef846a9189c50427e8: stopping tablet replica
I20260923 15:01:41.932241 15933 raft_consensus.cc:2229] T 68a83f9b46f04efea7c3139289aa7aeb P 3e1f390e2f6b43ef846a9189c50427e8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 15:01:41.932289 15933 raft_consensus.cc:2258] T 68a83f9b46f04efea7c3139289aa7aeb P 3e1f390e2f6b43ef846a9189c50427e8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 15:01:41.932334 15349 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46020:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 15:01:41.932483 15349 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 15:01:41.932560 15934 ts_tablet_manager.cc:1905] T 68a83f9b46f04efea7c3139289aa7aeb P 6e03dc9443b94cd09a97dc52085ffead: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 15:01:41.932699 15933 ts_tablet_manager.cc:1905] T 68a83f9b46f04efea7c3139289aa7aeb P 3e1f390e2f6b43ef846a9189c50427e8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 15:01:41.933902 15932 ts_tablet_manager.cc:1905] T 68a83f9b46f04efea7c3139289aa7aeb P 2cd60cb8df9646af83deb3452da89de2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 15:01:41.934144 15933 ts_tablet_manager.cc:1918] T 68a83f9b46f04efea7c3139289aa7aeb P 3e1f390e2f6b43ef846a9189c50427e8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260923 15:01:41.934224 15933 log.cc:1196] T 68a83f9b46f04efea7c3139289aa7aeb P 3e1f390e2f6b43ef846a9189c50427e8: Deleting WAL directory at /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175691491353-13614-0/minicluster-data/ts-1/wal/wals/68a83f9b46f04efea7c3139289aa7aeb
I20260923 15:01:41.934451 15933 ts_tablet_manager.cc:1939] T 68a83f9b46f04efea7c3139289aa7aeb P 3e1f390e2f6b43ef846a9189c50427e8: Deleting consensus metadata
I20260923 15:01:41.934829 15934 ts_tablet_manager.cc:1918] T 68a83f9b46f04efea7c3139289aa7aeb P 6e03dc9443b94cd09a97dc52085ffead: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260923 15:01:41.934914 15934 log.cc:1196] T 68a83f9b46f04efea7c3139289aa7aeb P 6e03dc9443b94cd09a97dc52085ffead: Deleting WAL directory at /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175691491353-13614-0/minicluster-data/ts-0/wal/wals/68a83f9b46f04efea7c3139289aa7aeb
I20260923 15:01:41.934998 15336 catalog_manager.cc:4841] TS 3e1f390e2f6b43ef846a9189c50427e8 (127.13.75.130:37633): tablet 68a83f9b46f04efea7c3139289aa7aeb (table test-table [id=f94bf9522b504e75b9c08554fff2e65c]) successfully deleted
I20260923 15:01:41.935142 15934 ts_tablet_manager.cc:1939] T 68a83f9b46f04efea7c3139289aa7aeb P 6e03dc9443b94cd09a97dc52085ffead: Deleting consensus metadata
I20260923 15:01:41.935307 15471 tablet_service.cc:1467] Processing CreateTablet for tablet 882d63f85fde413598b9f05d20914f8f (DEFAULT_TABLE table=test-table [id=ed095eacebb44129ae36f03c0a25bb37]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 15:01:41.935312 15626 tablet_service.cc:1467] Processing CreateTablet for tablet 882d63f85fde413598b9f05d20914f8f (DEFAULT_TABLE table=test-table [id=ed095eacebb44129ae36f03c0a25bb37]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 15:01:41.935408 15471 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 882d63f85fde413598b9f05d20914f8f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:01:41.935428 15626 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 882d63f85fde413598b9f05d20914f8f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:01:41.936655 15944 tablet_bootstrap.cc:492] T 882d63f85fde413598b9f05d20914f8f P 3e1f390e2f6b43ef846a9189c50427e8: Bootstrap starting.
I20260923 15:01:41.936807 15945 tablet_bootstrap.cc:492] T 882d63f85fde413598b9f05d20914f8f P 6e03dc9443b94cd09a97dc52085ffead: Bootstrap starting.
I20260923 15:01:41.937067 15766 tablet_service.cc:1467] Processing CreateTablet for tablet 882d63f85fde413598b9f05d20914f8f (DEFAULT_TABLE table=test-table [id=ed095eacebb44129ae36f03c0a25bb37]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 15:01:41.937108 15336 catalog_manager.cc:4841] TS 6e03dc9443b94cd09a97dc52085ffead (127.13.75.129:36423): tablet 68a83f9b46f04efea7c3139289aa7aeb (table test-table [id=f94bf9522b504e75b9c08554fff2e65c]) successfully deleted
I20260923 15:01:41.937186 15766 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 882d63f85fde413598b9f05d20914f8f. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:01:41.937211 15945 tablet_bootstrap.cc:654] T 882d63f85fde413598b9f05d20914f8f P 6e03dc9443b94cd09a97dc52085ffead: Neither blocks nor log segments found. Creating new log.
I20260923 15:01:41.937242 15944 tablet_bootstrap.cc:654] T 882d63f85fde413598b9f05d20914f8f P 3e1f390e2f6b43ef846a9189c50427e8: Neither blocks nor log segments found. Creating new log.
I20260923 15:01:41.937812 15944 tablet_bootstrap.cc:492] T 882d63f85fde413598b9f05d20914f8f P 3e1f390e2f6b43ef846a9189c50427e8: No bootstrap required, opened a new log
I20260923 15:01:41.937844 15944 ts_tablet_manager.cc:1397] T 882d63f85fde413598b9f05d20914f8f P 3e1f390e2f6b43ef846a9189c50427e8: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260923 15:01:41.937856 15932 ts_tablet_manager.cc:1918] T 68a83f9b46f04efea7c3139289aa7aeb P 2cd60cb8df9646af83deb3452da89de2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260923 15:01:41.937920 15932 log.cc:1196] T 68a83f9b46f04efea7c3139289aa7aeb P 2cd60cb8df9646af83deb3452da89de2: Deleting WAL directory at /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175691491353-13614-0/minicluster-data/ts-2/wal/wals/68a83f9b46f04efea7c3139289aa7aeb
I20260923 15:01:41.937947 15944 raft_consensus.cc:348] T 882d63f85fde413598b9f05d20914f8f P 3e1f390e2f6b43ef846a9189c50427e8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e03dc9443b94cd09a97dc52085ffead" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 36423 } } peers { permanent_uuid: "3e1f390e2f6b43ef846a9189c50427e8" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 37633 } } peers { permanent_uuid: "2cd60cb8df9646af83deb3452da89de2" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 43295 } }
I20260923 15:01:41.937991 15944 raft_consensus.cc:374] T 882d63f85fde413598b9f05d20914f8f P 3e1f390e2f6b43ef846a9189c50427e8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:01:41.938004 15944 raft_consensus.cc:729] T 882d63f85fde413598b9f05d20914f8f P 3e1f390e2f6b43ef846a9189c50427e8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e1f390e2f6b43ef846a9189c50427e8, State: Initialized, Role: FOLLOWER
I20260923 15:01:41.938067 15944 consensus_queue.cc:260] T 882d63f85fde413598b9f05d20914f8f P 3e1f390e2f6b43ef846a9189c50427e8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e03dc9443b94cd09a97dc52085ffead" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 36423 } } peers { permanent_uuid: "3e1f390e2f6b43ef846a9189c50427e8" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 37633 } } peers { permanent_uuid: "2cd60cb8df9646af83deb3452da89de2" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 43295 } }
I20260923 15:01:41.938125 15932 ts_tablet_manager.cc:1939] T 68a83f9b46f04efea7c3139289aa7aeb P 2cd60cb8df9646af83deb3452da89de2: Deleting consensus metadata
I20260923 15:01:41.938159 15944 ts_tablet_manager.cc:1428] T 882d63f85fde413598b9f05d20914f8f P 3e1f390e2f6b43ef846a9189c50427e8: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260923 15:01:41.938551 15333 catalog_manager.cc:4841] TS 2cd60cb8df9646af83deb3452da89de2 (127.13.75.131:43295): tablet 68a83f9b46f04efea7c3139289aa7aeb (table test-table [id=f94bf9522b504e75b9c08554fff2e65c]) successfully deleted
I20260923 15:01:41.938565 15943 tablet_bootstrap.cc:492] T 882d63f85fde413598b9f05d20914f8f P 2cd60cb8df9646af83deb3452da89de2: Bootstrap starting.
I20260923 15:01:41.938866 15943 tablet_bootstrap.cc:654] T 882d63f85fde413598b9f05d20914f8f P 2cd60cb8df9646af83deb3452da89de2: Neither blocks nor log segments found. Creating new log.
I20260923 15:01:41.939167 15945 tablet_bootstrap.cc:492] T 882d63f85fde413598b9f05d20914f8f P 6e03dc9443b94cd09a97dc52085ffead: No bootstrap required, opened a new log
I20260923 15:01:41.939220 15945 ts_tablet_manager.cc:1397] T 882d63f85fde413598b9f05d20914f8f P 6e03dc9443b94cd09a97dc52085ffead: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 15:01:41.939340 15945 raft_consensus.cc:348] T 882d63f85fde413598b9f05d20914f8f P 6e03dc9443b94cd09a97dc52085ffead [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e03dc9443b94cd09a97dc52085ffead" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 36423 } } peers { permanent_uuid: "3e1f390e2f6b43ef846a9189c50427e8" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 37633 } } peers { permanent_uuid: "2cd60cb8df9646af83deb3452da89de2" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 43295 } }
I20260923 15:01:41.939431 15945 raft_consensus.cc:374] T 882d63f85fde413598b9f05d20914f8f P 6e03dc9443b94cd09a97dc52085ffead [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:01:41.939456 15945 raft_consensus.cc:729] T 882d63f85fde413598b9f05d20914f8f P 6e03dc9443b94cd09a97dc52085ffead [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6e03dc9443b94cd09a97dc52085ffead, State: Initialized, Role: FOLLOWER
I20260923 15:01:41.939512 15943 tablet_bootstrap.cc:492] T 882d63f85fde413598b9f05d20914f8f P 2cd60cb8df9646af83deb3452da89de2: No bootstrap required, opened a new log
I20260923 15:01:41.939502 15945 consensus_queue.cc:260] T 882d63f85fde413598b9f05d20914f8f P 6e03dc9443b94cd09a97dc52085ffead [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e03dc9443b94cd09a97dc52085ffead" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 36423 } } peers { permanent_uuid: "3e1f390e2f6b43ef846a9189c50427e8" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 37633 } } peers { permanent_uuid: "2cd60cb8df9646af83deb3452da89de2" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 43295 } }
I20260923 15:01:41.939570 15943 ts_tablet_manager.cc:1397] T 882d63f85fde413598b9f05d20914f8f P 2cd60cb8df9646af83deb3452da89de2: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260923 15:01:41.939611 15945 ts_tablet_manager.cc:1428] T 882d63f85fde413598b9f05d20914f8f P 6e03dc9443b94cd09a97dc52085ffead: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260923 15:01:41.939674 15943 raft_consensus.cc:348] T 882d63f85fde413598b9f05d20914f8f P 2cd60cb8df9646af83deb3452da89de2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e03dc9443b94cd09a97dc52085ffead" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 36423 } } peers { permanent_uuid: "3e1f390e2f6b43ef846a9189c50427e8" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 37633 } } peers { permanent_uuid: "2cd60cb8df9646af83deb3452da89de2" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 43295 } }
I20260923 15:01:41.939742 15943 raft_consensus.cc:374] T 882d63f85fde413598b9f05d20914f8f P 2cd60cb8df9646af83deb3452da89de2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:01:41.939770 15943 raft_consensus.cc:729] T 882d63f85fde413598b9f05d20914f8f P 2cd60cb8df9646af83deb3452da89de2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2cd60cb8df9646af83deb3452da89de2, State: Initialized, Role: FOLLOWER
I20260923 15:01:41.939817 15943 consensus_queue.cc:260] T 882d63f85fde413598b9f05d20914f8f P 2cd60cb8df9646af83deb3452da89de2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e03dc9443b94cd09a97dc52085ffead" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 36423 } } peers { permanent_uuid: "3e1f390e2f6b43ef846a9189c50427e8" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 37633 } } peers { permanent_uuid: "2cd60cb8df9646af83deb3452da89de2" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 43295 } }
I20260923 15:01:41.939917 15943 ts_tablet_manager.cc:1428] T 882d63f85fde413598b9f05d20914f8f P 2cd60cb8df9646af83deb3452da89de2: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260923 15:01:42.181663 15926 raft_consensus.cc:482] T 882d63f85fde413598b9f05d20914f8f P 2cd60cb8df9646af83deb3452da89de2 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 15:01:42.181775 15926 raft_consensus.cc:504] T 882d63f85fde413598b9f05d20914f8f P 2cd60cb8df9646af83deb3452da89de2 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e03dc9443b94cd09a97dc52085ffead" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 36423 } } peers { permanent_uuid: "3e1f390e2f6b43ef846a9189c50427e8" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 37633 } } peers { permanent_uuid: "2cd60cb8df9646af83deb3452da89de2" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 43295 } }
I20260923 15:01:42.181923 15926 leader_election.cc:288] T 882d63f85fde413598b9f05d20914f8f P 2cd60cb8df9646af83deb3452da89de2 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 6e03dc9443b94cd09a97dc52085ffead (127.13.75.129:36423), 3e1f390e2f6b43ef846a9189c50427e8 (127.13.75.130:37633)
I20260923 15:01:42.182168 15497 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "882d63f85fde413598b9f05d20914f8f" candidate_uuid: "2cd60cb8df9646af83deb3452da89de2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e03dc9443b94cd09a97dc52085ffead" is_pre_election: true
I20260923 15:01:42.182284 15497 raft_consensus.cc:2454] T 882d63f85fde413598b9f05d20914f8f P 6e03dc9443b94cd09a97dc52085ffead [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2cd60cb8df9646af83deb3452da89de2 in term 0.
I20260923 15:01:42.182216 15646 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "882d63f85fde413598b9f05d20914f8f" candidate_uuid: "2cd60cb8df9646af83deb3452da89de2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e1f390e2f6b43ef846a9189c50427e8" is_pre_election: true
I20260923 15:01:42.182345 15646 raft_consensus.cc:2454] T 882d63f85fde413598b9f05d20914f8f P 3e1f390e2f6b43ef846a9189c50427e8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2cd60cb8df9646af83deb3452da89de2 in term 0.
I20260923 15:01:42.182528 15720 leader_election.cc:302] T 882d63f85fde413598b9f05d20914f8f P 2cd60cb8df9646af83deb3452da89de2 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2cd60cb8df9646af83deb3452da89de2, 6e03dc9443b94cd09a97dc52085ffead; no voters:
I20260923 15:01:42.182695 15926 raft_consensus.cc:2781] T 882d63f85fde413598b9f05d20914f8f P 2cd60cb8df9646af83deb3452da89de2 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 15:01:42.182742 15926 raft_consensus.cc:482] T 882d63f85fde413598b9f05d20914f8f P 2cd60cb8df9646af83deb3452da89de2 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 15:01:42.182772 15926 raft_consensus.cc:3037] T 882d63f85fde413598b9f05d20914f8f P 2cd60cb8df9646af83deb3452da89de2 [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:01:42.183562 15926 raft_consensus.cc:504] T 882d63f85fde413598b9f05d20914f8f P 2cd60cb8df9646af83deb3452da89de2 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e03dc9443b94cd09a97dc52085ffead" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 36423 } } peers { permanent_uuid: "3e1f390e2f6b43ef846a9189c50427e8" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 37633 } } peers { permanent_uuid: "2cd60cb8df9646af83deb3452da89de2" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 43295 } }
I20260923 15:01:42.183708 15926 leader_election.cc:288] T 882d63f85fde413598b9f05d20914f8f P 2cd60cb8df9646af83deb3452da89de2 [CANDIDATE]: Term 1 election: Requested vote from peers 6e03dc9443b94cd09a97dc52085ffead (127.13.75.129:36423), 3e1f390e2f6b43ef846a9189c50427e8 (127.13.75.130:37633)
I20260923 15:01:42.183907 15497 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "882d63f85fde413598b9f05d20914f8f" candidate_uuid: "2cd60cb8df9646af83deb3452da89de2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "6e03dc9443b94cd09a97dc52085ffead"
I20260923 15:01:42.183998 15497 raft_consensus.cc:3037] T 882d63f85fde413598b9f05d20914f8f P 6e03dc9443b94cd09a97dc52085ffead [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:01:42.183934 15646 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "882d63f85fde413598b9f05d20914f8f" candidate_uuid: "2cd60cb8df9646af83deb3452da89de2" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e1f390e2f6b43ef846a9189c50427e8"
I20260923 15:01:42.184046 15646 raft_consensus.cc:3037] T 882d63f85fde413598b9f05d20914f8f P 3e1f390e2f6b43ef846a9189c50427e8 [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:01:42.184612 15497 raft_consensus.cc:2454] T 882d63f85fde413598b9f05d20914f8f P 6e03dc9443b94cd09a97dc52085ffead [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2cd60cb8df9646af83deb3452da89de2 in term 1.
I20260923 15:01:42.184804 15646 raft_consensus.cc:2454] T 882d63f85fde413598b9f05d20914f8f P 3e1f390e2f6b43ef846a9189c50427e8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2cd60cb8df9646af83deb3452da89de2 in term 1.
I20260923 15:01:42.184824 15720 leader_election.cc:302] T 882d63f85fde413598b9f05d20914f8f P 2cd60cb8df9646af83deb3452da89de2 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2cd60cb8df9646af83deb3452da89de2, 6e03dc9443b94cd09a97dc52085ffead; no voters:
I20260923 15:01:42.184907 15926 raft_consensus.cc:2781] T 882d63f85fde413598b9f05d20914f8f P 2cd60cb8df9646af83deb3452da89de2 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 15:01:42.184955 15926 raft_consensus.cc:686] T 882d63f85fde413598b9f05d20914f8f P 2cd60cb8df9646af83deb3452da89de2 [term 1 LEADER]: Becoming Leader. State: Replica: 2cd60cb8df9646af83deb3452da89de2, State: Running, Role: LEADER
I20260923 15:01:42.185009 15926 consensus_queue.cc:237] T 882d63f85fde413598b9f05d20914f8f P 2cd60cb8df9646af83deb3452da89de2 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e03dc9443b94cd09a97dc52085ffead" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 36423 } } peers { permanent_uuid: "3e1f390e2f6b43ef846a9189c50427e8" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 37633 } } peers { permanent_uuid: "2cd60cb8df9646af83deb3452da89de2" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 43295 } }
I20260923 15:01:42.185623 15348 catalog_manager.cc:5495] T 882d63f85fde413598b9f05d20914f8f P 2cd60cb8df9646af83deb3452da89de2 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2cd60cb8df9646af83deb3452da89de2 (127.13.75.131). New cstate: current_term: 1 leader_uuid: "2cd60cb8df9646af83deb3452da89de2" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "6e03dc9443b94cd09a97dc52085ffead" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 36423 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "3e1f390e2f6b43ef846a9189c50427e8" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 37633 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2cd60cb8df9646af83deb3452da89de2" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 43295 } health_report { overall_health: HEALTHY } } }
I20260923 15:01:42.228109 15646 raft_consensus.cc:1261] T 882d63f85fde413598b9f05d20914f8f P 3e1f390e2f6b43ef846a9189c50427e8 [term 1 FOLLOWER]: Refusing update from remote peer 2cd60cb8df9646af83deb3452da89de2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 15:01:42.228132 15497 raft_consensus.cc:1261] T 882d63f85fde413598b9f05d20914f8f P 6e03dc9443b94cd09a97dc52085ffead [term 1 FOLLOWER]: Refusing update from remote peer 2cd60cb8df9646af83deb3452da89de2: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 15:01:42.228344 15923 consensus_queue.cc:1035] T 882d63f85fde413598b9f05d20914f8f P 2cd60cb8df9646af83deb3452da89de2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e1f390e2f6b43ef846a9189c50427e8" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 37633 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 15:01:42.228439 15926 consensus_queue.cc:1035] T 882d63f85fde413598b9f05d20914f8f P 2cd60cb8df9646af83deb3452da89de2 [LEADER]: Connected to new peer: Peer: permanent_uuid: "6e03dc9443b94cd09a97dc52085ffead" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 36423 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260923 15:01:42.232932 15914 tablet_replica.cc:1298] Aborted: operation has been aborted: cancelling pending write operations
I20260923 15:01:43.751058 15348 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46020:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 15:01:43.751171 15348 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:46020:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 15:01:43.752059 15348 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 413f7cf8ddb348c7a6a08580e4599d5b: Sending DeleteTablet for 3 replicas of tablet 882d63f85fde413598b9f05d20914f8f
I20260923 15:01:43.752372 15626 tablet_service.cc:1514] Processing DeleteTablet for tablet 882d63f85fde413598b9f05d20914f8f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 15:01:43 UTC) from {username='kudu', principal='kudu/127.13.75.190@KRBTEST.COM'} at 127.0.0.1:51474
I20260923 15:01:43.752381 15766 tablet_service.cc:1514] Processing DeleteTablet for tablet 882d63f85fde413598b9f05d20914f8f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 15:01:43 UTC) from {username='kudu', principal='kudu/127.13.75.190@KRBTEST.COM'} at 127.0.0.1:55714
I20260923 15:01:43.752579 15984 tablet_replica.cc:330] T 882d63f85fde413598b9f05d20914f8f P 3e1f390e2f6b43ef846a9189c50427e8: stopping tablet replica
I20260923 15:01:43.752563 13614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKC4soy/build/release/bin/kudu with pid 15421
I20260923 15:01:43.752591 15471 tablet_service.cc:1514] Processing DeleteTablet for tablet 882d63f85fde413598b9f05d20914f8f with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 15:01:43 UTC) from {username='kudu', principal='kudu/127.13.75.190@KRBTEST.COM'} at 127.0.0.1:40180
I20260923 15:01:43.753190 15984 raft_consensus.cc:2229] T 882d63f85fde413598b9f05d20914f8f P 3e1f390e2f6b43ef846a9189c50427e8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 15:01:43.753254 15984 raft_consensus.cc:2258] T 882d63f85fde413598b9f05d20914f8f P 3e1f390e2f6b43ef846a9189c50427e8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 15:01:43.753358 15985 tablet_replica.cc:330] T 882d63f85fde413598b9f05d20914f8f P 2cd60cb8df9646af83deb3452da89de2: stopping tablet replica
I20260923 15:01:43.753410 15985 raft_consensus.cc:2229] T 882d63f85fde413598b9f05d20914f8f P 2cd60cb8df9646af83deb3452da89de2 [term 1 LEADER]: Raft consensus shutting down.
I20260923 15:01:43.753468 15985 raft_consensus.cc:2258] T 882d63f85fde413598b9f05d20914f8f P 2cd60cb8df9646af83deb3452da89de2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 15:01:43.753592 15984 ts_tablet_manager.cc:1905] T 882d63f85fde413598b9f05d20914f8f P 3e1f390e2f6b43ef846a9189c50427e8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 15:01:43.753743 15985 ts_tablet_manager.cc:1905] T 882d63f85fde413598b9f05d20914f8f P 2cd60cb8df9646af83deb3452da89de2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 15:01:43.755244 15984 ts_tablet_manager.cc:1918] T 882d63f85fde413598b9f05d20914f8f P 3e1f390e2f6b43ef846a9189c50427e8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260923 15:01:43.755342 15984 log.cc:1196] T 882d63f85fde413598b9f05d20914f8f P 3e1f390e2f6b43ef846a9189c50427e8: Deleting WAL directory at /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175691491353-13614-0/minicluster-data/ts-1/wal/wals/882d63f85fde413598b9f05d20914f8f
I20260923 15:01:43.755631 15984 ts_tablet_manager.cc:1939] T 882d63f85fde413598b9f05d20914f8f P 3e1f390e2f6b43ef846a9189c50427e8: Deleting consensus metadata
I20260923 15:01:43.755628 15985 ts_tablet_manager.cc:1918] T 882d63f85fde413598b9f05d20914f8f P 2cd60cb8df9646af83deb3452da89de2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.8
I20260923 15:01:43.755710 15985 log.cc:1196] T 882d63f85fde413598b9f05d20914f8f P 2cd60cb8df9646af83deb3452da89de2: Deleting WAL directory at /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TxnSmokeWithDifferentUserTypes.1790175691491353-13614-0/minicluster-data/ts-2/wal/wals/882d63f85fde413598b9f05d20914f8f
I20260923 15:01:43.755978 15985 ts_tablet_manager.cc:1939] T 882d63f85fde413598b9f05d20914f8f P 2cd60cb8df9646af83deb3452da89de2: Deleting consensus metadata
I20260923 15:01:43.756009 15336 catalog_manager.cc:4841] TS 3e1f390e2f6b43ef846a9189c50427e8 (127.13.75.130:37633): tablet 882d63f85fde413598b9f05d20914f8f (table test-table [id=ed095eacebb44129ae36f03c0a25bb37]) successfully deleted
I20260923 15:01:43.756397 15333 catalog_manager.cc:4841] TS 2cd60cb8df9646af83deb3452da89de2 (127.13.75.131:43295): tablet 882d63f85fde413598b9f05d20914f8f (table test-table [id=ed095eacebb44129ae36f03c0a25bb37]) successfully deleted
W20260923 15:01:43.760776 15336 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.75.129:36423 (error 108)
I20260923 15:01:43.760836 13614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKC4soy/build/release/bin/kudu with pid 15561
I20260923 15:01:43.762132 15987 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:43.761861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.129:36423 (local address 127.0.0.1:49498)
0923 15:01:43.762016 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0923 15:01:43.762072 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.129:36423: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":86,"threads_started":1}
W20260923 15:01:43.762333 15336 catalog_manager.cc:4584] TS 6e03dc9443b94cd09a97dc52085ffead (127.13.75.129:36423): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 882d63f85fde413598b9f05d20914f8f: Network error: Client connection negotiation failed: client connection to 127.13.75.129:36423: connect: Connection refused (error 111)
I20260923 15:01:43.766706 13614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKC4soy/build/release/bin/kudu with pid 15701
I20260923 15:01:43.772708 13614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKC4soy/build/release/bin/kudu with pid 15317
2026-09-23T15:01:43Z chronyd exiting
[ OK ] SecurityITest.TxnSmokeWithDifferentUserTypes (5874 ms)
[ RUN ] SecurityITest.TestNoKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175691491353-13614-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 15:01:43 dist-test-slave-cnb0 krb5kdc[15990](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 15:01:43 dist-test-slave-cnb0 krb5kdc[15990](info): set up 2 sockets
Sep 23 15:01:43 dist-test-slave-cnb0 krb5kdc[15990](info): commencing operation
krb5kdc: starting...
W20260923 15:01:44.828123 13614 mini_kdc.cc:121] Time spent starting KDC: real 1.033s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175691491353-13614-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175691491353-13614-0/krb5kdc/test-user.keytab.
Sep 23 15:01:44 dist-test-slave-cnb0 krb5kdc[15990](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175704, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T15:01:44Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T15:01:44Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.75.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.75.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.75.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.190.keytab.
Entry for principal kudu/127.13.75.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.190.keytab.
I20260923 15:01:44.963923 13614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKC4soy/build/release/bin/kudu
/tmp/dist-test-taskKC4soy/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175691491353-13614-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175691491353-13614-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175691491353-13614-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175691491353-13614-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.75.190:46547
--webserver_interface=127.13.75.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.190.keytab
--principal=kudu/127.13.75.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.75.148:34445
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.75.190:46547
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175691491353-13614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175691491353-13614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175691491353-13614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:01:45.053094 16003 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:01:45.053417 16003 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:01:45.053478 16003 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:01:45.054754 16003 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 15:01:45.054857 16003 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:01:45.054885 16003 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:01:45.054939 16003 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 15:01:45.054970 16003 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 15:01:45.056650 16003 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.75.148:34445
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175691491353-13614-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175691491353-13614-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.75.190:46547
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.190.keytab
--principal=kudu/127.13.75.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.75.190:46547
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175691491353-13614-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.75.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16003
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175691491353-13614-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:01:45.057008 16003 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:01:45.057287 16003 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:01:45.062999 16011 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:01:45.062999 16008 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:01:45.062999 16009 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:01:45.063247 16003 server_base.cc:1034] running on GCE node
I20260923 15:01:45.063573 16003 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:01:45.063949 16003 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:01:45.065157 16003 hybrid_clock.cc:648] HybridClock initialized: now 1790175705065119 us; error 55 us; skew 500 ppm
Sep 23 15:01:45 dist-test-slave-cnb0 krb5kdc[15990](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175705, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:01:45.068347 16003 init.cc:377] Logged in from keytab as kudu/127.13.75.190@KRBTEST.COM (short username kudu)
I20260923 15:01:45.068883 16003 webserver.cc:460] Webserver started at http://127.13.75.190:43511/ using document root <none> and password file <none>
I20260923 15:01:45.069164 16003 fs_manager.cc:359] Metadata directory not provided
I20260923 15:01:45.069211 16003 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:01:45.069360 16003 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:01:45.070499 16003 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175691491353-13614-0/minicluster-data/master-0/data/instance:
uuid: "44be3ecfeac040c3814d998a86c02a9e"
format_stamp: "Formatted at 2026-09-23 15:01:45 on dist-test-slave-cnb0"
server_key: "047305867e582e128fc52dfd692e85ff"
server_key_iv: "87ffbb3cfe32b97e380ad1bc3dd33213"
server_key_version: "encryptionkey@0"
I20260923 15:01:45.070937 16003 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175691491353-13614-0/minicluster-data/master-0/wal/instance:
uuid: "44be3ecfeac040c3814d998a86c02a9e"
format_stamp: "Formatted at 2026-09-23 15:01:45 on dist-test-slave-cnb0"
server_key: "047305867e582e128fc52dfd692e85ff"
server_key_iv: "87ffbb3cfe32b97e380ad1bc3dd33213"
server_key_version: "encryptionkey@0"
I20260923 15:01:45.072481 16003 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260923 15:01:45.073388 16018 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:01:45.073736 16003 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.001s
I20260923 15:01:45.073832 16003 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175691491353-13614-0/minicluster-data/master-0/data,/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175691491353-13614-0/minicluster-data/master-0/wal
uuid: "44be3ecfeac040c3814d998a86c02a9e"
format_stamp: "Formatted at 2026-09-23 15:01:45 on dist-test-slave-cnb0"
server_key: "047305867e582e128fc52dfd692e85ff"
server_key_iv: "87ffbb3cfe32b97e380ad1bc3dd33213"
server_key_version: "encryptionkey@0"
I20260923 15:01:45.073892 16003 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175691491353-13614-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175691491353-13614-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175691491353-13614-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:01:45.091962 16003 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:01:45.094513 16003 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:01:45.094648 16003 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:01:45.098869 16003 rpc_server.cc:307] RPC server started. Bound to: 127.13.75.190:46547
I20260923 15:01:45.099053 16070 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.75.190:46547 every 8 connection(s)
I20260923 15:01:45.099403 16003 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175691491353-13614-0/minicluster-data/master-0/data/info.pb
I20260923 15:01:45.099920 13614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKC4soy/build/release/bin/kudu as pid 16003
I20260923 15:01:45.100029 13614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175691491353-13614-0/minicluster-data/master-0/wal/instance
I20260923 15:01:45.100214 13614 external_mini_cluster.cc:1428] Setting key 2e592fac54720438a5ef07d74304afd5
I20260923 15:01:45.100637 16071 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:01:45.104367 16071 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 44be3ecfeac040c3814d998a86c02a9e: Bootstrap starting.
I20260923 15:01:45.105122 16071 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 44be3ecfeac040c3814d998a86c02a9e: Neither blocks nor log segments found. Creating new log.
I20260923 15:01:45.105437 16071 log.cc:824] T 00000000000000000000000000000000 P 44be3ecfeac040c3814d998a86c02a9e: Log is configured to *not* fsync() on all Append() calls
I20260923 15:01:45.106389 16071 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 44be3ecfeac040c3814d998a86c02a9e: No bootstrap required, opened a new log
Sep 23 15:01:45 dist-test-slave-cnb0 krb5kdc[15990](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175704, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.75.190@KRBTEST.COM
I20260923 15:01:45.108968 16071 raft_consensus.cc:348] T 00000000000000000000000000000000 P 44be3ecfeac040c3814d998a86c02a9e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44be3ecfeac040c3814d998a86c02a9e" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 46547 } }
I20260923 15:01:45.109202 16071 raft_consensus.cc:374] T 00000000000000000000000000000000 P 44be3ecfeac040c3814d998a86c02a9e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:01:45.109243 16071 raft_consensus.cc:729] T 00000000000000000000000000000000 P 44be3ecfeac040c3814d998a86c02a9e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 44be3ecfeac040c3814d998a86c02a9e, State: Initialized, Role: FOLLOWER
I20260923 15:01:45.109393 16071 consensus_queue.cc:260] T 00000000000000000000000000000000 P 44be3ecfeac040c3814d998a86c02a9e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44be3ecfeac040c3814d998a86c02a9e" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 46547 } }
I20260923 15:01:45.109505 16071 raft_consensus.cc:388] T 00000000000000000000000000000000 P 44be3ecfeac040c3814d998a86c02a9e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 15:01:45.109552 16071 raft_consensus.cc:482] T 00000000000000000000000000000000 P 44be3ecfeac040c3814d998a86c02a9e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 15:01:45.109625 16071 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 44be3ecfeac040c3814d998a86c02a9e [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:01:45.110457 16071 raft_consensus.cc:504] T 00000000000000000000000000000000 P 44be3ecfeac040c3814d998a86c02a9e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44be3ecfeac040c3814d998a86c02a9e" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 46547 } }
I20260923 15:01:45.110584 16071 leader_election.cc:302] T 00000000000000000000000000000000 P 44be3ecfeac040c3814d998a86c02a9e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 44be3ecfeac040c3814d998a86c02a9e; no voters:
I20260923 15:01:45.110795 16071 leader_election.cc:288] T 00000000000000000000000000000000 P 44be3ecfeac040c3814d998a86c02a9e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 15:01:45.110918 16076 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 44be3ecfeac040c3814d998a86c02a9e [term 1 FOLLOWER]: Leader election won for term 1
I20260923 15:01:45.111092 16076 raft_consensus.cc:686] T 00000000000000000000000000000000 P 44be3ecfeac040c3814d998a86c02a9e [term 1 LEADER]: Becoming Leader. State: Replica: 44be3ecfeac040c3814d998a86c02a9e, State: Running, Role: LEADER
I20260923 15:01:45.111191 16071 sys_catalog.cc:564] T 00000000000000000000000000000000 P 44be3ecfeac040c3814d998a86c02a9e [sys.catalog]: configured and running, proceeding with master startup.
I20260923 15:01:45.111198 16076 consensus_queue.cc:237] T 00000000000000000000000000000000 P 44be3ecfeac040c3814d998a86c02a9e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44be3ecfeac040c3814d998a86c02a9e" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 46547 } }
I20260923 15:01:45.111614 16076 sys_catalog.cc:455] T 00000000000000000000000000000000 P 44be3ecfeac040c3814d998a86c02a9e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "44be3ecfeac040c3814d998a86c02a9e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44be3ecfeac040c3814d998a86c02a9e" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 46547 } } }
I20260923 15:01:45.111804 16076 sys_catalog.cc:458] T 00000000000000000000000000000000 P 44be3ecfeac040c3814d998a86c02a9e [sys.catalog]: This master's current role is: LEADER
I20260923 15:01:45.112172 16076 sys_catalog.cc:455] T 00000000000000000000000000000000 P 44be3ecfeac040c3814d998a86c02a9e [sys.catalog]: SysCatalogTable state changed. Reason: New leader 44be3ecfeac040c3814d998a86c02a9e. Latest consensus state: current_term: 1 leader_uuid: "44be3ecfeac040c3814d998a86c02a9e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "44be3ecfeac040c3814d998a86c02a9e" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 46547 } } }
I20260923 15:01:45.112319 16076 sys_catalog.cc:458] T 00000000000000000000000000000000 P 44be3ecfeac040c3814d998a86c02a9e [sys.catalog]: This master's current role is: LEADER
I20260923 15:01:45.112732 16085 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 15:01:45.113103 16074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:45.100812 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49546 (local address 127.13.75.190:46547)
0923 15:01:45.100993 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0923 15:01:45.101002 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0923 15:01:45.101035 (+ 33us) server_negotiation.cc:407] Connection header received
0923 15:01:45.101503 (+ 468us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:01:45.101510 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:01:45.101559 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:01:45.101680 (+ 121us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:01:45.102517 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:45.103524 (+ 1007us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:45.104594 (+ 1070us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:45.105219 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:45.105349 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:01:45.108343 (+ 2994us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:01:45.108350 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:01:45.108351 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:01:45.108378 (+ 27us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:01:45.110743 (+ 2365us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:01:45.111248 (+ 505us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:01:45.111252 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:01:45.111252 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:01:45.111364 (+ 112us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:01:45.111616 (+ 252us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:01:45.111617 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:01:45.111618 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:01:45.112121 (+ 503us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:01:45.112295 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0923 15:01:45.112724 (+ 429us) server_negotiation.cc:299] Negotiation successful
0923 15:01:45.112880 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":125,"thread_start_us":93,"threads_started":1}
I20260923 15:01:45.113485 16085 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 15:01:45.115079 16085 catalog_manager.cc:1347] Generated new cluster ID: 2530b3fd63d24c628bd3af64a7925961
I20260923 15:01:45.115135 16085 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 15:01:45.131668 16085 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 15:01:45.132555 16085 catalog_manager.cc:1504] Loading token signing keys...
I20260923 15:01:45.139302 16085 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 44be3ecfeac040c3814d998a86c02a9e: Generated new TSK 0
I20260923 15:01:45.139568 16085 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.75.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.75.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.75.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.129.keytab.
Entry for principal kudu/127.13.75.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.129.keytab.
I20260923 15:01:45.175881 13614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKC4soy/build/release/bin/kudu
/tmp/dist-test-taskKC4soy/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175691491353-13614-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175691491353-13614-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175691491353-13614-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175691491353-13614-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.75.129:0
--local_ip_for_outbound_sockets=127.13.75.129
--webserver_interface=127.13.75.129
--webserver_port=0
--tserver_master_addrs=127.13.75.190:46547
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.129.keytab
--principal=kudu/127.13.75.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.75.148:34445
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175691491353-13614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175691491353-13614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175691491353-13614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:01:45.276150 16097 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:01:45.276525 16097 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:01:45.276582 16097 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:01:45.279325 16097 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:01:45.279417 16097 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:01:45.279474 16097 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.75.129
I20260923 15:01:45.282943 16097 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.75.148:34445
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175691491353-13614-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175691491353-13614-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.129.keytab
--principal=kudu/127.13.75.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.75.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175691491353-13614-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.75.129
--webserver_port=0
--tserver_master_addrs=127.13.75.190:46547
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16097
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.75.129
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175691491353-13614-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:01:45.283337 16097 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:01:45.283712 16097 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:01:45.290968 16102 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:01:45.291105 16105 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:01:45.291052 16103 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:01:45.291448 16097 server_base.cc:1034] running on GCE node
I20260923 15:01:45.291702 16097 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:01:45.291982 16097 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:01:45.293241 16097 hybrid_clock.cc:648] HybridClock initialized: now 1790175705293217 us; error 34 us; skew 500 ppm
Sep 23 15:01:45 dist-test-slave-cnb0 krb5kdc[15990](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175705, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:01:45.296461 16097 init.cc:377] Logged in from keytab as kudu/127.13.75.129@KRBTEST.COM (short username kudu)
I20260923 15:01:45.296886 16097 webserver.cc:460] Webserver started at http://127.13.75.129:38417/ using document root <none> and password file <none>
I20260923 15:01:45.297144 16097 fs_manager.cc:359] Metadata directory not provided
I20260923 15:01:45.297192 16097 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:01:45.297339 16097 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:01:45.298460 16097 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175691491353-13614-0/minicluster-data/ts-0/data/instance:
uuid: "cdfefdd0f17e4c71952ff6e1069d2c81"
format_stamp: "Formatted at 2026-09-23 15:01:45 on dist-test-slave-cnb0"
server_key: "93ed565b57c2fd9361b337764e801259"
server_key_iv: "ee3800993c1abae4844ad265797c006a"
server_key_version: "encryptionkey@0"
I20260923 15:01:45.298841 16097 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175691491353-13614-0/minicluster-data/ts-0/wal/instance:
uuid: "cdfefdd0f17e4c71952ff6e1069d2c81"
format_stamp: "Formatted at 2026-09-23 15:01:45 on dist-test-slave-cnb0"
server_key: "93ed565b57c2fd9361b337764e801259"
server_key_iv: "ee3800993c1abae4844ad265797c006a"
server_key_version: "encryptionkey@0"
I20260923 15:01:45.300400 16097 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260923 15:01:45.301429 16112 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:01:45.301767 16097 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:01:45.301885 16097 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175691491353-13614-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175691491353-13614-0/minicluster-data/ts-0/wal
uuid: "cdfefdd0f17e4c71952ff6e1069d2c81"
format_stamp: "Formatted at 2026-09-23 15:01:45 on dist-test-slave-cnb0"
server_key: "93ed565b57c2fd9361b337764e801259"
server_key_iv: "ee3800993c1abae4844ad265797c006a"
server_key_version: "encryptionkey@0"
I20260923 15:01:45.301963 16097 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175691491353-13614-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175691491353-13614-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175691491353-13614-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:01:45.341609 16097 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:01:45.343952 16097 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:01:45.344107 16097 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:01:45.344378 16097 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 15:01:45.344714 16097 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 15:01:45.344749 16097 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 15:01:45.344772 16097 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 15:01:45.344785 16097 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 15:01:45.350833 16097 rpc_server.cc:307] RPC server started. Bound to: 127.13.75.129:40457
I20260923 15:01:45.350876 16225 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.75.129:40457 every 8 connection(s)
I20260923 15:01:45.351301 16097 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175691491353-13614-0/minicluster-data/ts-0/data/info.pb
I20260923 15:01:45.352629 13614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKC4soy/build/release/bin/kudu as pid 16097
I20260923 15:01:45.352718 13614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175691491353-13614-0/minicluster-data/ts-0/wal/instance
I20260923 15:01:45.352897 13614 external_mini_cluster.cc:1428] Setting key b9c77c717de8d7b94b991d5c64aa3873
Sep 23 15:01:45 dist-test-slave-cnb0 krb5kdc[15990](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175705, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.129@KRBTEST.COM for kudu/127.13.75.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.13.75.130@KRBTEST.COM; defaulting to no policy
I20260923 15:01:45.363554 16074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:45.352322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.129:58859 (local address 127.13.75.190:46547)
0923 15:01:45.352394 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0923 15:01:45.352396 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:01:45.352523 (+ 127us) server_negotiation.cc:407] Connection header received
0923 15:01:45.353282 (+ 759us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:01:45.353283 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:01:45.353295 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:01:45.353378 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:01:45.354508 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:45.355344 (+ 836us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:45.356972 (+ 1628us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:45.357172 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:45.357233 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:01:45.360205 (+ 2972us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:01:45.360207 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:01:45.360207 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:01:45.360216 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:01:45.361738 (+ 1522us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:01:45.362315 (+ 577us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:01:45.362316 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:01:45.362317 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:01:45.362378 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:01:45.362706 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:01:45.362708 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:01:45.362708 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:01:45.362874 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:01:45.363020 (+ 146us) server_negotiation.cc:1035] Waiting for connection context
0923 15:01:45.363353 (+ 333us) server_negotiation.cc:299] Negotiation successful
0923 15:01:45.363466 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260923 15:01:45.363624 16228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:45.352202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:46547 (local address 127.13.75.129:58859)
0923 15:01:45.352378 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0923 15:01:45.352423 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0923 15:01:45.353142 (+ 719us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:01:45.353487 (+ 345us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:01:45.353492 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:01:45.354187 (+ 695us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:01:45.354389 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:01:45.354394 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:01:45.355457 (+ 1063us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:01:45.355458 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:01:45.356869 (+ 1411us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:01:45.356872 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:01:45.357272 (+ 400us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:01:45.357274 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:01:45.357432 (+ 158us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:01:45.358112 (+ 680us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:01:45.358123 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:01:45.360077 (+ 1954us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:01:45.361891 (+ 1814us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:01:45.361900 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:01:45.361904 (+ 4us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:01:45.362213 (+ 309us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:01:45.362459 (+ 246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:01:45.362460 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:01:45.362460 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:01:45.362596 (+ 136us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:01:45.363028 (+ 432us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:01:45.363036 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:01:45.363202 (+ 166us) client_negotiation.cc:769] Sending connection context
0923 15:01:45.363300 (+ 98us) client_negotiation.cc:240] Negotiation successful
0923 15:01:45.363435 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":105,"threads_started":1}
I20260923 15:01:45.364284 16226 heartbeater.cc:347] Connected to a master server at 127.13.75.190:46547
I20260923 15:01:45.364377 16226 heartbeater.cc:464] Registering TS with master...
I20260923 15:01:45.364533 16226 heartbeater.cc:511] Master 127.13.75.190:46547 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.75.130@KRBTEST.COM" created.
I20260923 15:01:45.365144 16035 ts_manager.cc:194] Registered new tserver with Master: cdfefdd0f17e4c71952ff6e1069d2c81 (127.13.75.129:40457)
I20260923 15:01:45.365808 16035 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.75.129@KRBTEST.COM'} at 127.13.75.129:58859
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.75.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.130.keytab.
Entry for principal kudu/127.13.75.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.130.keytab.
I20260923 15:01:45.383607 13614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKC4soy/build/release/bin/kudu
/tmp/dist-test-taskKC4soy/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175691491353-13614-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175691491353-13614-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175691491353-13614-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175691491353-13614-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.75.130:0
--local_ip_for_outbound_sockets=127.13.75.130
--webserver_interface=127.13.75.130
--webserver_port=0
--tserver_master_addrs=127.13.75.190:46547
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.130.keytab
--principal=kudu/127.13.75.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.75.148:34445
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175691491353-13614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175691491353-13614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175691491353-13614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:01:45.467316 16231 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:01:45.467545 16231 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:01:45.467571 16231 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:01:45.468724 16231 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:01:45.468761 16231 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:01:45.468789 16231 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.75.130
I20260923 15:01:45.470225 16231 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.75.148:34445
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175691491353-13614-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175691491353-13614-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.130.keytab
--principal=kudu/127.13.75.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.75.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175691491353-13614-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.75.130
--webserver_port=0
--tserver_master_addrs=127.13.75.190:46547
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16231
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.75.130
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175691491353-13614-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:01:45.470432 16231 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:01:45.470625 16231 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:01:45.476173 16236 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:01:45.476179 16239 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:01:45.476187 16237 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:01:45.476656 16231 server_base.cc:1034] running on GCE node
I20260923 15:01:45.476877 16231 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:01:45.477149 16231 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:01:45.478324 16231 hybrid_clock.cc:648] HybridClock initialized: now 1790175705478309 us; error 25 us; skew 500 ppm
Sep 23 15:01:45 dist-test-slave-cnb0 krb5kdc[15990](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175705, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:01:45.481056 16231 init.cc:377] Logged in from keytab as kudu/127.13.75.130@KRBTEST.COM (short username kudu)
I20260923 15:01:45.481585 16231 webserver.cc:460] Webserver started at http://127.13.75.130:35725/ using document root <none> and password file <none>
I20260923 15:01:45.481833 16231 fs_manager.cc:359] Metadata directory not provided
I20260923 15:01:45.481906 16231 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:01:45.482062 16231 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:01:45.483242 16231 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175691491353-13614-0/minicluster-data/ts-1/data/instance:
uuid: "c36f9c5abeb94243b00630a60cf340f6"
format_stamp: "Formatted at 2026-09-23 15:01:45 on dist-test-slave-cnb0"
server_key: "3bc694891c108a03766a6ae5f654564a"
server_key_iv: "10da1f34941452154f913c212918f337"
server_key_version: "encryptionkey@0"
I20260923 15:01:45.483628 16231 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175691491353-13614-0/minicluster-data/ts-1/wal/instance:
uuid: "c36f9c5abeb94243b00630a60cf340f6"
format_stamp: "Formatted at 2026-09-23 15:01:45 on dist-test-slave-cnb0"
server_key: "3bc694891c108a03766a6ae5f654564a"
server_key_iv: "10da1f34941452154f913c212918f337"
server_key_version: "encryptionkey@0"
I20260923 15:01:45.485070 16231 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 15:01:45.485948 16246 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:01:45.486145 16231 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:01:45.486258 16231 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175691491353-13614-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175691491353-13614-0/minicluster-data/ts-1/wal
uuid: "c36f9c5abeb94243b00630a60cf340f6"
format_stamp: "Formatted at 2026-09-23 15:01:45 on dist-test-slave-cnb0"
server_key: "3bc694891c108a03766a6ae5f654564a"
server_key_iv: "10da1f34941452154f913c212918f337"
server_key_version: "encryptionkey@0"
I20260923 15:01:45.486336 16231 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175691491353-13614-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175691491353-13614-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175691491353-13614-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:01:45.498980 16231 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:01:45.501282 16231 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:01:45.501477 16231 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:01:45.501742 16231 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 15:01:45.502096 16231 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 15:01:45.502137 16231 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 15:01:45.502161 16231 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 15:01:45.502174 16231 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 15:01:45.508873 16231 rpc_server.cc:307] RPC server started. Bound to: 127.13.75.130:40299
I20260923 15:01:45.508931 16359 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.75.130:40299 every 8 connection(s)
I20260923 15:01:45.509256 16231 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175691491353-13614-0/minicluster-data/ts-1/data/info.pb
Sep 23 15:01:45 dist-test-slave-cnb0 krb5kdc[15990](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175705, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.130@KRBTEST.COM for kudu/127.13.75.190@KRBTEST.COM
I20260923 15:01:45.519192 13614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKC4soy/build/release/bin/kudu as pid 16231
I20260923 15:01:45.519312 13614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175691491353-13614-0/minicluster-data/ts-1/wal/instance
I20260923 15:01:45.519523 13614 external_mini_cluster.cc:1428] Setting key 11ecbea3363aa0295c4040cfdc7e7c60
I20260923 15:01:45.520469 16074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:45.510067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.130:45699 (local address 127.13.75.190:46547)
0923 15:01:45.510165 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0923 15:01:45.510167 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:01:45.510634 (+ 467us) server_negotiation.cc:407] Connection header received
0923 15:01:45.511395 (+ 761us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:01:45.511396 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:01:45.511410 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:01:45.511452 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:01:45.512811 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:45.513384 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:45.514499 (+ 1115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:45.514724 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:45.514824 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:01:45.517205 (+ 2381us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:01:45.517207 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:01:45.517208 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:01:45.517217 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:01:45.518696 (+ 1479us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:01:45.519322 (+ 626us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:01:45.519323 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:01:45.519323 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:01:45.519390 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:01:45.519772 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:01:45.519773 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:01:45.519773 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:01:45.519901 (+ 128us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:01:45.519987 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0923 15:01:45.520283 (+ 296us) server_negotiation.cc:299] Negotiation successful
0923 15:01:45.520366 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":62}
I20260923 15:01:45.520673 16362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:45.510223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:46547 (local address 127.13.75.130:45699)
0923 15:01:45.510461 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0923 15:01:45.510509 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0923 15:01:45.511253 (+ 744us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:01:45.511609 (+ 356us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:01:45.511613 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:01:45.512446 (+ 833us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:01:45.512677 (+ 231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:01:45.512683 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:01:45.513497 (+ 814us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:01:45.513499 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:01:45.514387 (+ 888us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:01:45.514390 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:01:45.514836 (+ 446us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:01:45.514837 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:01:45.514998 (+ 161us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:01:45.515584 (+ 586us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:01:45.515593 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:01:45.517091 (+ 1498us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:01:45.518862 (+ 1771us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:01:45.518870 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:01:45.518872 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:01:45.519221 (+ 349us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:01:45.519531 (+ 310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:01:45.519532 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:01:45.519532 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:01:45.519693 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:01:45.519977 (+ 284us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:01:45.519984 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:01:45.520143 (+ 159us) client_negotiation.cc:769] Sending connection context
0923 15:01:45.520250 (+ 107us) client_negotiation.cc:240] Negotiation successful
0923 15:01:45.520416 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":86,"threads_started":1}
I20260923 15:01:45.521217 16360 heartbeater.cc:347] Connected to a master server at 127.13.75.190:46547
I20260923 15:01:45.521337 16360 heartbeater.cc:464] Registering TS with master...
I20260923 15:01:45.521548 16360 heartbeater.cc:511] Master 127.13.75.190:46547 requested a full tablet report, sending...
I20260923 15:01:45.521983 16035 ts_manager.cc:194] Registered new tserver with Master: c36f9c5abeb94243b00630a60cf340f6 (127.13.75.130:40299)
I20260923 15:01:45.522385 16035 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.75.130@KRBTEST.COM'} at 127.13.75.130:45699
WARNING: no policy specified for kudu/127.13.75.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.75.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.75.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.131.keytab.
Entry for principal kudu/127.13.75.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.131.keytab.
I20260923 15:01:45.551237 13614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKC4soy/build/release/bin/kudu
/tmp/dist-test-taskKC4soy/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175691491353-13614-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175691491353-13614-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175691491353-13614-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175691491353-13614-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.75.131:0
--local_ip_for_outbound_sockets=127.13.75.131
--webserver_interface=127.13.75.131
--webserver_port=0
--tserver_master_addrs=127.13.75.190:46547
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.131.keytab
--principal=kudu/127.13.75.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.75.148:34445
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175691491353-13614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175691491353-13614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175691491353-13614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:01:45.637018 16365 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:01:45.637229 16365 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:01:45.637254 16365 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:01:45.638366 16365 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:01:45.638401 16365 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:01:45.638427 16365 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.75.131
I20260923 15:01:45.640132 16365 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.75.148:34445
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175691491353-13614-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175691491353-13614-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.131.keytab
--principal=kudu/127.13.75.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.75.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175691491353-13614-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.75.131
--webserver_port=0
--tserver_master_addrs=127.13.75.190:46547
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16365
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.75.131
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175691491353-13614-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:01:45.640341 16365 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:01:45.640533 16365 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:01:45.646020 16373 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:01:45.646020 16371 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:01:45.646046 16370 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:01:45.646339 16365 server_base.cc:1034] running on GCE node
I20260923 15:01:45.646589 16365 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:01:45.646849 16365 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:01:45.648067 16365 hybrid_clock.cc:648] HybridClock initialized: now 1790175705648039 us; error 41 us; skew 500 ppm
Sep 23 15:01:45 dist-test-slave-cnb0 krb5kdc[15990](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175705, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:01:45.651304 16365 init.cc:377] Logged in from keytab as kudu/127.13.75.131@KRBTEST.COM (short username kudu)
I20260923 15:01:45.651937 16365 webserver.cc:460] Webserver started at http://127.13.75.131:37681/ using document root <none> and password file <none>
I20260923 15:01:45.652240 16365 fs_manager.cc:359] Metadata directory not provided
I20260923 15:01:45.652329 16365 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:01:45.652491 16365 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:01:45.653877 16365 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175691491353-13614-0/minicluster-data/ts-2/data/instance:
uuid: "b08ddab21fd04ded8d7840248aeb90d6"
format_stamp: "Formatted at 2026-09-23 15:01:45 on dist-test-slave-cnb0"
server_key: "5d42200c0ffa8f6c0f933f11b8650f85"
server_key_iv: "380c2d5dc0096f7e417a9336572ae8e0"
server_key_version: "encryptionkey@0"
I20260923 15:01:45.654390 16365 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175691491353-13614-0/minicluster-data/ts-2/wal/instance:
uuid: "b08ddab21fd04ded8d7840248aeb90d6"
format_stamp: "Formatted at 2026-09-23 15:01:45 on dist-test-slave-cnb0"
server_key: "5d42200c0ffa8f6c0f933f11b8650f85"
server_key_iv: "380c2d5dc0096f7e417a9336572ae8e0"
server_key_version: "encryptionkey@0"
I20260923 15:01:45.656085 16365 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.004s
I20260923 15:01:45.656903 16380 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:01:45.657172 16365 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260923 15:01:45.657248 16365 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175691491353-13614-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175691491353-13614-0/minicluster-data/ts-2/wal
uuid: "b08ddab21fd04ded8d7840248aeb90d6"
format_stamp: "Formatted at 2026-09-23 15:01:45 on dist-test-slave-cnb0"
server_key: "5d42200c0ffa8f6c0f933f11b8650f85"
server_key_iv: "380c2d5dc0096f7e417a9336572ae8e0"
server_key_version: "encryptionkey@0"
I20260923 15:01:45.657302 16365 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175691491353-13614-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175691491353-13614-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175691491353-13614-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:01:45.678076 16365 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:01:45.680601 16365 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:01:45.680718 16365 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:01:45.680915 16365 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 15:01:45.681253 16365 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 15:01:45.681290 16365 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 15:01:45.681314 16365 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 15:01:45.681327 16365 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 15:01:45.687471 16365 rpc_server.cc:307] RPC server started. Bound to: 127.13.75.131:42755
I20260923 15:01:45.687507 16493 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.75.131:42755 every 8 connection(s)
I20260923 15:01:45.687865 16365 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175691491353-13614-0/minicluster-data/ts-2/data/info.pb
Sep 23 15:01:45 dist-test-slave-cnb0 krb5kdc[15990](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175705, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.131@KRBTEST.COM for kudu/127.13.75.190@KRBTEST.COM
I20260923 15:01:45.696708 13614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKC4soy/build/release/bin/kudu as pid 16365
I20260923 15:01:45.696827 13614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNoKerberosCredentials.1790175691491353-13614-0/minicluster-data/ts-2/wal/instance
I20260923 15:01:45.697009 13614 external_mini_cluster.cc:1428] Setting key 77680a2625d0a54625b9153b924f25af
I20260923 15:01:45.698376 16074 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:45.688674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.131:58195 (local address 127.13.75.190:46547)
0923 15:01:45.688749 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0923 15:01:45.688772 (+ 23us) server_negotiation.cc:399] Waiting for connection header
0923 15:01:45.688976 (+ 204us) server_negotiation.cc:407] Connection header received
0923 15:01:45.689686 (+ 710us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:01:45.689688 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:01:45.689700 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:01:45.689754 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:01:45.690860 (+ 1106us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:45.691621 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:45.692728 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:45.692947 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:45.693043 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:01:45.695284 (+ 2241us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:01:45.695286 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:01:45.695286 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:01:45.695293 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:01:45.696707 (+ 1414us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:01:45.697223 (+ 516us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:01:45.697224 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:01:45.697224 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:01:45.697290 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:01:45.697679 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:01:45.697680 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:01:45.697681 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:01:45.697813 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:01:45.697897 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0923 15:01:45.698200 (+ 303us) server_negotiation.cc:299] Negotiation successful
0923 15:01:45.698283 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260923 15:01:45.698588 16496 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:45.688644 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:46547 (local address 127.13.75.131:58195)
0923 15:01:45.688844 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0923 15:01:45.688885 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0923 15:01:45.689570 (+ 685us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:01:45.689838 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:01:45.689842 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:01:45.690506 (+ 664us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:01:45.690759 (+ 253us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:01:45.690764 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:01:45.691773 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:01:45.691775 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:01:45.692604 (+ 829us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:01:45.692607 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:01:45.693060 (+ 453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:01:45.693061 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:01:45.693190 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:01:45.693716 (+ 526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:01:45.693725 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:01:45.695192 (+ 1467us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:01:45.696801 (+ 1609us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:01:45.696810 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:01:45.696812 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:01:45.697137 (+ 325us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:01:45.697426 (+ 289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:01:45.697427 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:01:45.697428 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:01:45.697602 (+ 174us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:01:45.697888 (+ 286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:01:45.697894 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:01:45.698051 (+ 157us) client_negotiation.cc:769] Sending connection context
0923 15:01:45.698168 (+ 117us) client_negotiation.cc:240] Negotiation successful
0923 15:01:45.698341 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":92,"threads_started":1}
I20260923 15:01:45.699119 16494 heartbeater.cc:347] Connected to a master server at 127.13.75.190:46547
I20260923 15:01:45.699239 16494 heartbeater.cc:464] Registering TS with master...
I20260923 15:01:45.699427 16494 heartbeater.cc:511] Master 127.13.75.190:46547 requested a full tablet report, sending...
I20260923 15:01:45.699834 16035 ts_manager.cc:194] Registered new tserver with Master: b08ddab21fd04ded8d7840248aeb90d6 (127.13.75.131:42755)
I20260923 15:01:45.700233 16035 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.75.131@KRBTEST.COM'} at 127.13.75.131:58195
I20260923 15:01:45.709151 13614 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260923 15:01:45.721261 16074 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 15:01:45.720103 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49552 (local address 127.13.75.190:46547)
0923 15:01:45.720220 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0923 15:01:45.720223 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:01:45.720345 (+ 122us) server_negotiation.cc:407] Connection header received
0923 15:01:45.720452 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:01:45.720453 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:01:45.720465 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:01:45.720548 (+ 83us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:01:45.721196 (+ 648us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:49552: BlockingRecv error: recv got EOF from 127.0.0.1:49552 (error 108)
Metrics: {"server-negotiator.queue_time_us":80}
I20260923 15:01:45.722402 13614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKC4soy/build/release/bin/kudu with pid 16097
I20260923 15:01:45.726941 13614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKC4soy/build/release/bin/kudu with pid 16231
I20260923 15:01:45.731602 13614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKC4soy/build/release/bin/kudu with pid 16365
I20260923 15:01:45.735962 13614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKC4soy/build/release/bin/kudu with pid 16003
2026-09-23T15:01:45Z chronyd exiting
[ OK ] SecurityITest.TestNoKerberosCredentials (1961 ms)
[ RUN ] SecurityITest.SaslPlainFallback
Loading random data
Initializing database '/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790175691491353-13614-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 15:01:45 dist-test-slave-cnb0 krb5kdc[16506](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 15:01:45 dist-test-slave-cnb0 krb5kdc[16506](info): set up 2 sockets
Sep 23 15:01:45 dist-test-slave-cnb0 krb5kdc[16506](info): commencing operation
krb5kdc: starting...
W20260923 15:01:46.840483 13614 mini_kdc.cc:121] Time spent starting KDC: real 1.084s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790175691491353-13614-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790175691491353-13614-0/krb5kdc/test-user.keytab.
Sep 23 15:01:46 dist-test-slave-cnb0 krb5kdc[16506](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175706, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T15:01:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T15:01:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.75.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.75.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.75.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.190.keytab.
Entry for principal kudu/127.13.75.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.190.keytab.
I20260923 15:01:46.976812 13614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKC4soy/build/release/bin/kudu
/tmp/dist-test-taskKC4soy/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790175691491353-13614-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790175691491353-13614-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790175691491353-13614-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790175691491353-13614-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.75.190:34541
--webserver_interface=127.13.75.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.190.keytab
--principal=kudu/127.13.75.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.75.148:41141
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.75.190:34541
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc-authentication=optional
--user-acl=* with env {KRB5CCNAME=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790175691491353-13614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790175691491353-13614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790175691491353-13614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:01:47.064371 16519 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:01:47.064631 16519 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:01:47.064674 16519 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:01:47.065874 16519 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 15:01:47.065953 16519 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:01:47.065979 16519 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:01:47.066001 16519 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 15:01:47.066031 16519 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 15:01:47.067595 16519 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.75.148:41141
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790175691491353-13614-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790175691491353-13614-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.75.190:34541
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.190.keytab
--principal=kudu/127.13.75.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.75.190:34541
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790175691491353-13614-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.75.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16519
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790175691491353-13614-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:01:47.067890 16519 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:01:47.068140 16519 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:01:47.073096 16527 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:01:47.073187 16525 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:01:47.073104 16524 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:01:47.073308 16519 server_base.cc:1034] running on GCE node
I20260923 15:01:47.073596 16519 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:01:47.073853 16519 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:01:47.075034 16519 hybrid_clock.cc:648] HybridClock initialized: now 1790175707075031 us; error 43 us; skew 500 ppm
Sep 23 15:01:47 dist-test-slave-cnb0 krb5kdc[16506](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175707, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:01:47.080920 16519 init.cc:377] Logged in from keytab as kudu/127.13.75.190@KRBTEST.COM (short username kudu)
I20260923 15:01:47.081420 16519 webserver.cc:460] Webserver started at http://127.13.75.190:45755/ using document root <none> and password file <none>
I20260923 15:01:47.081693 16519 fs_manager.cc:359] Metadata directory not provided
I20260923 15:01:47.081741 16519 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:01:47.081899 16519 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:01:47.083103 16519 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790175691491353-13614-0/minicluster-data/master-0/data/instance:
uuid: "3ce01d1c25ad4ce5aa89bf97f4ebd2cb"
format_stamp: "Formatted at 2026-09-23 15:01:47 on dist-test-slave-cnb0"
server_key: "01538607a54beaea99551aa9eafd4c39"
server_key_iv: "2995e1625000e49d75c02f1e8a165368"
server_key_version: "encryptionkey@0"
I20260923 15:01:47.083487 16519 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790175691491353-13614-0/minicluster-data/master-0/wal/instance:
uuid: "3ce01d1c25ad4ce5aa89bf97f4ebd2cb"
format_stamp: "Formatted at 2026-09-23 15:01:47 on dist-test-slave-cnb0"
server_key: "01538607a54beaea99551aa9eafd4c39"
server_key_iv: "2995e1625000e49d75c02f1e8a165368"
server_key_version: "encryptionkey@0"
I20260923 15:01:47.084983 16519 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260923 15:01:47.085983 16534 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:01:47.086228 16519 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260923 15:01:47.086325 16519 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790175691491353-13614-0/minicluster-data/master-0/data,/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790175691491353-13614-0/minicluster-data/master-0/wal
uuid: "3ce01d1c25ad4ce5aa89bf97f4ebd2cb"
format_stamp: "Formatted at 2026-09-23 15:01:47 on dist-test-slave-cnb0"
server_key: "01538607a54beaea99551aa9eafd4c39"
server_key_iv: "2995e1625000e49d75c02f1e8a165368"
server_key_version: "encryptionkey@0"
I20260923 15:01:47.086386 16519 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790175691491353-13614-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790175691491353-13614-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790175691491353-13614-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:01:47.108868 16519 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:01:47.111014 16519 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:01:47.111153 16519 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:01:47.115145 16519 rpc_server.cc:307] RPC server started. Bound to: 127.13.75.190:34541
I20260923 15:01:47.115191 16586 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.75.190:34541 every 8 connection(s)
I20260923 15:01:47.115674 16519 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790175691491353-13614-0/minicluster-data/master-0/data/info.pb
I20260923 15:01:47.116499 16587 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:01:47.118973 16587 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3ce01d1c25ad4ce5aa89bf97f4ebd2cb: Bootstrap starting.
I20260923 15:01:47.119607 16587 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3ce01d1c25ad4ce5aa89bf97f4ebd2cb: Neither blocks nor log segments found. Creating new log.
I20260923 15:01:47.119931 16587 log.cc:824] T 00000000000000000000000000000000 P 3ce01d1c25ad4ce5aa89bf97f4ebd2cb: Log is configured to *not* fsync() on all Append() calls
I20260923 15:01:47.120805 16587 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3ce01d1c25ad4ce5aa89bf97f4ebd2cb: No bootstrap required, opened a new log
I20260923 15:01:47.122557 16587 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3ce01d1c25ad4ce5aa89bf97f4ebd2cb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ce01d1c25ad4ce5aa89bf97f4ebd2cb" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 34541 } }
I20260923 15:01:47.122746 13614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKC4soy/build/release/bin/kudu as pid 16519
I20260923 15:01:47.122753 16587 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3ce01d1c25ad4ce5aa89bf97f4ebd2cb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:01:47.122800 16587 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3ce01d1c25ad4ce5aa89bf97f4ebd2cb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3ce01d1c25ad4ce5aa89bf97f4ebd2cb, State: Initialized, Role: FOLLOWER
I20260923 15:01:47.122866 13614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.SaslPlainFallback.1790175691491353-13614-0/minicluster-data/master-0/wal/instance
I20260923 15:01:47.123066 13614 external_mini_cluster.cc:1428] Setting key 2b79ac2d8f61c0c0b37f3083c0d76613
I20260923 15:01:47.123027 16587 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3ce01d1c25ad4ce5aa89bf97f4ebd2cb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ce01d1c25ad4ce5aa89bf97f4ebd2cb" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 34541 } }
I20260923 15:01:47.123145 16587 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3ce01d1c25ad4ce5aa89bf97f4ebd2cb [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 15:01:47.123195 16587 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3ce01d1c25ad4ce5aa89bf97f4ebd2cb [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 15:01:47.123250 16587 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3ce01d1c25ad4ce5aa89bf97f4ebd2cb [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:01:47.123979 16587 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3ce01d1c25ad4ce5aa89bf97f4ebd2cb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ce01d1c25ad4ce5aa89bf97f4ebd2cb" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 34541 } }
I20260923 15:01:47.124125 16587 leader_election.cc:302] T 00000000000000000000000000000000 P 3ce01d1c25ad4ce5aa89bf97f4ebd2cb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3ce01d1c25ad4ce5aa89bf97f4ebd2cb; no voters:
I20260923 15:01:47.124327 16587 leader_election.cc:288] T 00000000000000000000000000000000 P 3ce01d1c25ad4ce5aa89bf97f4ebd2cb [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 15:01:47.124368 16592 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3ce01d1c25ad4ce5aa89bf97f4ebd2cb [term 1 FOLLOWER]: Leader election won for term 1
I20260923 15:01:47.124472 16592 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3ce01d1c25ad4ce5aa89bf97f4ebd2cb [term 1 LEADER]: Becoming Leader. State: Replica: 3ce01d1c25ad4ce5aa89bf97f4ebd2cb, State: Running, Role: LEADER
I20260923 15:01:47.124570 16592 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3ce01d1c25ad4ce5aa89bf97f4ebd2cb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ce01d1c25ad4ce5aa89bf97f4ebd2cb" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 34541 } }
I20260923 15:01:47.124604 16587 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3ce01d1c25ad4ce5aa89bf97f4ebd2cb [sys.catalog]: configured and running, proceeding with master startup.
I20260923 15:01:47.124912 16593 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3ce01d1c25ad4ce5aa89bf97f4ebd2cb [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3ce01d1c25ad4ce5aa89bf97f4ebd2cb. Latest consensus state: current_term: 1 leader_uuid: "3ce01d1c25ad4ce5aa89bf97f4ebd2cb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ce01d1c25ad4ce5aa89bf97f4ebd2cb" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 34541 } } }
I20260923 15:01:47.125202 16593 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3ce01d1c25ad4ce5aa89bf97f4ebd2cb [sys.catalog]: This master's current role is: LEADER
I20260923 15:01:47.125121 16594 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3ce01d1c25ad4ce5aa89bf97f4ebd2cb [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3ce01d1c25ad4ce5aa89bf97f4ebd2cb" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3ce01d1c25ad4ce5aa89bf97f4ebd2cb" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 34541 } } }
I20260923 15:01:47.125541 16594 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3ce01d1c25ad4ce5aa89bf97f4ebd2cb [sys.catalog]: This master's current role is: LEADER
I20260923 15:01:47.125931 16601 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 15:01:47.126463 16601 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 15:01:47.128881 16601 catalog_manager.cc:1347] Generated new cluster ID: 37f23dd250764517ad9e794fda402caf
I20260923 15:01:47.128949 16601 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 23 15:01:47 dist-test-slave-cnb0 krb5kdc[16506](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175706, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.75.190@KRBTEST.COM
I20260923 15:01:47.136634 16591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:47.124074 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38730 (local address 127.13.75.190:34541)
0923 15:01:47.124349 (+ 275us) server_negotiation.cc:206] Beginning negotiation
0923 15:01:47.124359 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0923 15:01:47.124427 (+ 68us) server_negotiation.cc:407] Connection header received
0923 15:01:47.124570 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:01:47.124579 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:01:47.124653 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:01:47.125713 (+ 1060us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:01:47.125844 (+ 131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:47.127459 (+ 1615us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:47.128480 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:47.129198 (+ 718us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:47.129313 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:01:47.132467 (+ 3154us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:01:47.132477 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:01:47.132478 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:01:47.132493 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:01:47.134739 (+ 2246us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:01:47.135215 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:01:47.135218 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:01:47.135219 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:01:47.135348 (+ 129us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:01:47.135602 (+ 254us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:01:47.135602 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:01:47.135603 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:01:47.135946 (+ 343us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:01:47.136113 (+ 167us) server_negotiation.cc:1035] Waiting for connection context
0923 15:01:47.136229 (+ 116us) server_negotiation.cc:299] Negotiation successful
0923 15:01:47.136383 (+ 154us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":214,"thread_start_us":155,"threads_started":1}
I20260923 15:01:47.137742 13614 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 15:01:47.152724 16601 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 15:01:47.153609 16601 catalog_manager.cc:1504] Loading token signing keys...
I20260923 15:01:47.159466 16591 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:47.154789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38746 (local address 127.13.75.190:34541)
0923 15:01:47.154851 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0923 15:01:47.154853 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:01:47.154999 (+ 146us) server_negotiation.cc:407] Connection header received
0923 15:01:47.155053 (+ 54us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:01:47.155054 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:01:47.155066 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:01:47.155146 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:01:47.155618 (+ 472us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:47.156429 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:47.157658 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:47.158012 (+ 354us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:47.158090 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:01:47.158930 (+ 840us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:01:47.158931 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:01:47.158932 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 15:01:47.159169 (+ 237us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:01:47.159189 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 15:01:47.159197 (+ 8us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:01:47.159258 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0923 15:01:47.159349 (+ 91us) server_negotiation.cc:299] Negotiation successful
0923 15:01:47.159376 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260923 15:01:47.166160 16601 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3ce01d1c25ad4ce5aa89bf97f4ebd2cb: Generated new TSK 0
I20260923 15:01:47.166379 16601 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 15:01:47.172078 13614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKC4soy/build/release/bin/kudu with pid 16519
2026-09-23T15:01:47Z chronyd exiting
[ OK ] SecurityITest.SaslPlainFallback (1433 ms)
[ RUN ] SecurityITest.TestUnauthorizedClientKerberosCredentials
Loading random data
Initializing database '/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175691491353-13614-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 15:01:47 dist-test-slave-cnb0 krb5kdc[16620](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 15:01:47 dist-test-slave-cnb0 krb5kdc[16620](info): set up 2 sockets
Sep 23 15:01:47 dist-test-slave-cnb0 krb5kdc[16620](info): commencing operation
krb5kdc: starting...
W20260923 15:01:48.236896 13614 mini_kdc.cc:121] Time spent starting KDC: real 1.046s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175691491353-13614-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175691491353-13614-0/krb5kdc/test-user.keytab.
Sep 23 15:01:48 dist-test-slave-cnb0 krb5kdc[16620](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175708, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T15:01:48Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T15:01:48Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.75.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.75.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.75.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.190.keytab.
Entry for principal kudu/127.13.75.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.190.keytab.
I20260923 15:01:48.366801 13614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKC4soy/build/release/bin/kudu
/tmp/dist-test-taskKC4soy/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175691491353-13614-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175691491353-13614-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175691491353-13614-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175691491353-13614-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.75.190:34765
--webserver_interface=127.13.75.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.190.keytab
--principal=kudu/127.13.75.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.75.148:46759
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.75.190:34765
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175691491353-13614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175691491353-13614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175691491353-13614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:01:48.450531 16633 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:01:48.450827 16633 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:01:48.450872 16633 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:01:48.452448 16633 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 15:01:48.452574 16633 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:01:48.452611 16633 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:01:48.452643 16633 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 15:01:48.452677 16633 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 15:01:48.454605 16633 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.75.148:46759
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175691491353-13614-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175691491353-13614-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.75.190:34765
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.190.keytab
--principal=kudu/127.13.75.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.75.190:34765
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175691491353-13614-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.75.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16633
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175691491353-13614-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:01:48.455014 16633 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:01:48.455322 16633 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:01:48.461335 16638 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:01:48.461324 16639 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:01:48.461354 16633 server_base.cc:1034] running on GCE node
W20260923 15:01:48.461324 16641 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:01:48.462044 16633 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:01:48.462378 16633 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:01:48.463549 16633 hybrid_clock.cc:648] HybridClock initialized: now 1790175708463521 us; error 50 us; skew 500 ppm
Sep 23 15:01:48 dist-test-slave-cnb0 krb5kdc[16620](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175708, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:01:48.466487 16633 init.cc:377] Logged in from keytab as kudu/127.13.75.190@KRBTEST.COM (short username kudu)
I20260923 15:01:48.467126 16633 webserver.cc:460] Webserver started at http://127.13.75.190:41345/ using document root <none> and password file <none>
I20260923 15:01:48.467425 16633 fs_manager.cc:359] Metadata directory not provided
I20260923 15:01:48.467509 16633 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:01:48.467676 16633 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:01:48.468909 16633 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175691491353-13614-0/minicluster-data/master-0/data/instance:
uuid: "df3d290f057e4bccab669f975dcfa563"
format_stamp: "Formatted at 2026-09-23 15:01:48 on dist-test-slave-cnb0"
server_key: "5625a513fd7a2ce9470a58f6efa023e0"
server_key_iv: "fe58405fcf726aa6a2b56c10b5ceb0e2"
server_key_version: "encryptionkey@0"
I20260923 15:01:48.469303 16633 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175691491353-13614-0/minicluster-data/master-0/wal/instance:
uuid: "df3d290f057e4bccab669f975dcfa563"
format_stamp: "Formatted at 2026-09-23 15:01:48 on dist-test-slave-cnb0"
server_key: "5625a513fd7a2ce9470a58f6efa023e0"
server_key_iv: "fe58405fcf726aa6a2b56c10b5ceb0e2"
server_key_version: "encryptionkey@0"
I20260923 15:01:48.471004 16633 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.004s sys 0.000s
I20260923 15:01:48.472007 16648 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:01:48.472216 16633 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:01:48.472307 16633 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175691491353-13614-0/minicluster-data/master-0/data,/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175691491353-13614-0/minicluster-data/master-0/wal
uuid: "df3d290f057e4bccab669f975dcfa563"
format_stamp: "Formatted at 2026-09-23 15:01:48 on dist-test-slave-cnb0"
server_key: "5625a513fd7a2ce9470a58f6efa023e0"
server_key_iv: "fe58405fcf726aa6a2b56c10b5ceb0e2"
server_key_version: "encryptionkey@0"
I20260923 15:01:48.472361 16633 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175691491353-13614-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175691491353-13614-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175691491353-13614-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:01:48.488709 16633 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:01:48.491227 16633 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:01:48.491353 16633 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:01:48.495757 16633 rpc_server.cc:307] RPC server started. Bound to: 127.13.75.190:34765
I20260923 15:01:48.495931 16700 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.75.190:34765 every 8 connection(s)
I20260923 15:01:48.496222 16633 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175691491353-13614-0/minicluster-data/master-0/data/info.pb
I20260923 15:01:48.497294 16701 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:01:48.499768 16701 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P df3d290f057e4bccab669f975dcfa563: Bootstrap starting.
I20260923 15:01:48.500381 16701 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P df3d290f057e4bccab669f975dcfa563: Neither blocks nor log segments found. Creating new log.
I20260923 15:01:48.500690 16701 log.cc:824] T 00000000000000000000000000000000 P df3d290f057e4bccab669f975dcfa563: Log is configured to *not* fsync() on all Append() calls
I20260923 15:01:48.501591 16701 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P df3d290f057e4bccab669f975dcfa563: No bootstrap required, opened a new log
I20260923 15:01:48.502524 13614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKC4soy/build/release/bin/kudu as pid 16633
I20260923 15:01:48.502655 13614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175691491353-13614-0/minicluster-data/master-0/wal/instance
I20260923 15:01:48.502851 13614 external_mini_cluster.cc:1428] Setting key 7c0f8f39d75006c36d2072dcc58a09ca
I20260923 15:01:48.503963 16701 raft_consensus.cc:348] T 00000000000000000000000000000000 P df3d290f057e4bccab669f975dcfa563 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df3d290f057e4bccab669f975dcfa563" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 34765 } }
I20260923 15:01:48.504113 16701 raft_consensus.cc:374] T 00000000000000000000000000000000 P df3d290f057e4bccab669f975dcfa563 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:01:48.504171 16701 raft_consensus.cc:729] T 00000000000000000000000000000000 P df3d290f057e4bccab669f975dcfa563 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: df3d290f057e4bccab669f975dcfa563, State: Initialized, Role: FOLLOWER
I20260923 15:01:48.504406 16701 consensus_queue.cc:260] T 00000000000000000000000000000000 P df3d290f057e4bccab669f975dcfa563 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df3d290f057e4bccab669f975dcfa563" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 34765 } }
I20260923 15:01:48.504522 16701 raft_consensus.cc:388] T 00000000000000000000000000000000 P df3d290f057e4bccab669f975dcfa563 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 15:01:48.504575 16701 raft_consensus.cc:482] T 00000000000000000000000000000000 P df3d290f057e4bccab669f975dcfa563 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 15:01:48.504630 16701 raft_consensus.cc:3037] T 00000000000000000000000000000000 P df3d290f057e4bccab669f975dcfa563 [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:01:48.505621 16701 raft_consensus.cc:504] T 00000000000000000000000000000000 P df3d290f057e4bccab669f975dcfa563 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df3d290f057e4bccab669f975dcfa563" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 34765 } }
I20260923 15:01:48.505762 16701 leader_election.cc:302] T 00000000000000000000000000000000 P df3d290f057e4bccab669f975dcfa563 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: df3d290f057e4bccab669f975dcfa563; no voters:
I20260923 15:01:48.505982 16701 leader_election.cc:288] T 00000000000000000000000000000000 P df3d290f057e4bccab669f975dcfa563 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 15:01:48.506008 16706 raft_consensus.cc:2781] T 00000000000000000000000000000000 P df3d290f057e4bccab669f975dcfa563 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 15:01:48.506217 16706 raft_consensus.cc:686] T 00000000000000000000000000000000 P df3d290f057e4bccab669f975dcfa563 [term 1 LEADER]: Becoming Leader. State: Replica: df3d290f057e4bccab669f975dcfa563, State: Running, Role: LEADER
I20260923 15:01:48.506374 16701 sys_catalog.cc:564] T 00000000000000000000000000000000 P df3d290f057e4bccab669f975dcfa563 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 15:01:48.506366 16706 consensus_queue.cc:237] T 00000000000000000000000000000000 P df3d290f057e4bccab669f975dcfa563 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df3d290f057e4bccab669f975dcfa563" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 34765 } }
I20260923 15:01:48.508188 16707 sys_catalog.cc:455] T 00000000000000000000000000000000 P df3d290f057e4bccab669f975dcfa563 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "df3d290f057e4bccab669f975dcfa563" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df3d290f057e4bccab669f975dcfa563" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 34765 } } }
I20260923 15:01:48.508225 16708 sys_catalog.cc:455] T 00000000000000000000000000000000 P df3d290f057e4bccab669f975dcfa563 [sys.catalog]: SysCatalogTable state changed. Reason: New leader df3d290f057e4bccab669f975dcfa563. Latest consensus state: current_term: 1 leader_uuid: "df3d290f057e4bccab669f975dcfa563" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "df3d290f057e4bccab669f975dcfa563" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 34765 } } }
I20260923 15:01:48.508370 16707 sys_catalog.cc:458] T 00000000000000000000000000000000 P df3d290f057e4bccab669f975dcfa563 [sys.catalog]: This master's current role is: LEADER
I20260923 15:01:48.508525 16708 sys_catalog.cc:458] T 00000000000000000000000000000000 P df3d290f057e4bccab669f975dcfa563 [sys.catalog]: This master's current role is: LEADER
W20260923 15:01:48.508550 16721 catalog_manager.cc:1558] T 00000000000000000000000000000000 P df3d290f057e4bccab669f975dcfa563: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260923 15:01:48.508589 16721 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260923 15:01:48.508749 16723 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 15:01:48.508962 16723 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Sep 23 15:01:48 dist-test-slave-cnb0 krb5kdc[16620](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175708, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.75.190@KRBTEST.COM
I20260923 15:01:48.510636 16723 catalog_manager.cc:1347] Generated new cluster ID: f2cbdf368fe54b5798f76af5fd1c436c
I20260923 15:01:48.510696 16723 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 15:01:48.514619 16705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:48.503485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47350 (local address 127.13.75.190:34765)
0923 15:01:48.503702 (+ 217us) server_negotiation.cc:206] Beginning negotiation
0923 15:01:48.503711 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0923 15:01:48.503742 (+ 31us) server_negotiation.cc:407] Connection header received
0923 15:01:48.503908 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:01:48.503913 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:01:48.503960 (+ 47us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:01:48.504147 (+ 187us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:01:48.504959 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:48.505973 (+ 1014us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:48.506917 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:48.507592 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:48.507708 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:01:48.510615 (+ 2907us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:01:48.510619 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:01:48.510620 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:01:48.510633 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:01:48.512778 (+ 2145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:01:48.513224 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:01:48.513227 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:01:48.513227 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:01:48.513364 (+ 137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:01:48.513606 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:01:48.513607 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:01:48.513608 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:01:48.513934 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:01:48.514103 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0923 15:01:48.514204 (+ 101us) server_negotiation.cc:299] Negotiation successful
0923 15:01:48.514381 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":2,"server-negotiator.queue_time_us":156,"thread_start_us":99,"threads_started":1}
I20260923 15:01:48.517292 16723 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 15:01:48.518010 16723 catalog_manager.cc:1504] Loading token signing keys...
I20260923 15:01:48.527273 16723 catalog_manager.cc:5868] T 00000000000000000000000000000000 P df3d290f057e4bccab669f975dcfa563: Generated new TSK 0
I20260923 15:01:48.527504 16723 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.75.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.75.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.75.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.129.keytab.
Entry for principal kudu/127.13.75.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.129.keytab.
I20260923 15:01:48.558954 13614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKC4soy/build/release/bin/kudu
/tmp/dist-test-taskKC4soy/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175691491353-13614-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175691491353-13614-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175691491353-13614-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175691491353-13614-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.75.129:0
--local_ip_for_outbound_sockets=127.13.75.129
--webserver_interface=127.13.75.129
--webserver_port=0
--tserver_master_addrs=127.13.75.190:34765
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.129.keytab
--principal=kudu/127.13.75.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.75.148:46759
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175691491353-13614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175691491353-13614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175691491353-13614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:01:48.643800 16727 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:01:48.644021 16727 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:01:48.644047 16727 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:01:48.645269 16727 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:01:48.645320 16727 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:01:48.645349 16727 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.75.129
I20260923 15:01:48.646875 16727 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.75.148:46759
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175691491353-13614-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175691491353-13614-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.129.keytab
--principal=kudu/127.13.75.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.75.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175691491353-13614-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.75.129
--webserver_port=0
--tserver_master_addrs=127.13.75.190:34765
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16727
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.75.129
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175691491353-13614-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:01:48.647197 16727 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:01:48.647462 16727 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:01:48.653115 16735 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:01:48.653119 16732 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:01:48.653152 16727 server_base.cc:1034] running on GCE node
W20260923 15:01:48.653115 16733 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:01:48.653669 16727 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:01:48.653928 16727 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:01:48.655114 16727 hybrid_clock.cc:648] HybridClock initialized: now 1790175708655097 us; error 30 us; skew 500 ppm
Sep 23 15:01:48 dist-test-slave-cnb0 krb5kdc[16620](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175708, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:01:48.657893 16727 init.cc:377] Logged in from keytab as kudu/127.13.75.129@KRBTEST.COM (short username kudu)
I20260923 15:01:48.658382 16727 webserver.cc:460] Webserver started at http://127.13.75.129:46849/ using document root <none> and password file <none>
I20260923 15:01:48.658583 16727 fs_manager.cc:359] Metadata directory not provided
I20260923 15:01:48.658628 16727 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:01:48.658720 16727 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:01:48.659866 16727 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175691491353-13614-0/minicluster-data/ts-0/data/instance:
uuid: "e6f43c46fa14475493585425be7d6459"
format_stamp: "Formatted at 2026-09-23 15:01:48 on dist-test-slave-cnb0"
server_key: "0ab294836f7ffa24736819f72332e767"
server_key_iv: "9ee41933f0a5f1e15db641238110f77e"
server_key_version: "encryptionkey@0"
I20260923 15:01:48.660178 16727 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175691491353-13614-0/minicluster-data/ts-0/wal/instance:
uuid: "e6f43c46fa14475493585425be7d6459"
format_stamp: "Formatted at 2026-09-23 15:01:48 on dist-test-slave-cnb0"
server_key: "0ab294836f7ffa24736819f72332e767"
server_key_iv: "9ee41933f0a5f1e15db641238110f77e"
server_key_version: "encryptionkey@0"
I20260923 15:01:48.661494 16727 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 15:01:48.662298 16742 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:01:48.662563 16727 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:01:48.662669 16727 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175691491353-13614-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175691491353-13614-0/minicluster-data/ts-0/wal
uuid: "e6f43c46fa14475493585425be7d6459"
format_stamp: "Formatted at 2026-09-23 15:01:48 on dist-test-slave-cnb0"
server_key: "0ab294836f7ffa24736819f72332e767"
server_key_iv: "9ee41933f0a5f1e15db641238110f77e"
server_key_version: "encryptionkey@0"
I20260923 15:01:48.662746 16727 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175691491353-13614-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175691491353-13614-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175691491353-13614-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:01:48.690146 16727 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:01:48.692518 16727 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:01:48.692647 16727 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:01:48.692848 16727 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 15:01:48.693192 16727 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 15:01:48.693226 16727 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 15:01:48.693248 16727 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 15:01:48.693301 16727 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 15:01:48.699100 16727 rpc_server.cc:307] RPC server started. Bound to: 127.13.75.129:33199
I20260923 15:01:48.699193 16855 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.75.129:33199 every 8 connection(s)
I20260923 15:01:48.699514 16727 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175691491353-13614-0/minicluster-data/ts-0/data/info.pb
I20260923 15:01:48.704677 13614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKC4soy/build/release/bin/kudu as pid 16727
I20260923 15:01:48.704792 13614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175691491353-13614-0/minicluster-data/ts-0/wal/instance
I20260923 15:01:48.705060 13614 external_mini_cluster.cc:1428] Setting key 2098bea94555d00e594233dd0918cd4d
Sep 23 15:01:48 dist-test-slave-cnb0 krb5kdc[16620](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175708, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.129@KRBTEST.COM for kudu/127.13.75.190@KRBTEST.COM
I20260923 15:01:48.711424 16705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:48.700599 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.129:44883 (local address 127.13.75.190:34765)
0923 15:01:48.700685 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0923 15:01:48.700687 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:01:48.700926 (+ 239us) server_negotiation.cc:407] Connection header received
0923 15:01:48.701451 (+ 525us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:01:48.701453 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:01:48.701463 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:01:48.701531 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:01:48.702500 (+ 969us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:48.703034 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:48.704098 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:48.704311 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:48.704443 (+ 132us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:01:48.707647 (+ 3204us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:01:48.707649 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:01:48.707649 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:01:48.707660 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:01:48.709399 (+ 1739us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:01:48.710026 (+ 627us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:01:48.710027 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:01:48.710028 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:01:48.710094 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:01:48.710460 (+ 366us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:01:48.710461 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:01:48.710462 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:01:48.710613 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:01:48.710680 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0923 15:01:48.711254 (+ 574us) server_negotiation.cc:299] Negotiation successful
0923 15:01:48.711353 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260923 15:01:48.711712 16858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:48.700518 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:34765 (local address 127.13.75.129:44883)
0923 15:01:48.700756 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0923 15:01:48.700807 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0923 15:01:48.701286 (+ 479us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:01:48.701589 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:01:48.701592 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:01:48.702170 (+ 578us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:01:48.702368 (+ 198us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:01:48.702382 (+ 14us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:01:48.703119 (+ 737us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:01:48.703120 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:01:48.703988 (+ 868us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:01:48.703990 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:01:48.704510 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:01:48.704511 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:01:48.704693 (+ 182us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:01:48.705383 (+ 690us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:01:48.705394 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:01:48.707485 (+ 2091us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:01:48.709516 (+ 2031us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:01:48.709525 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:01:48.709528 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:01:48.709905 (+ 377us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:01:48.710188 (+ 283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:01:48.710189 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:01:48.710189 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:01:48.710369 (+ 180us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:01:48.710702 (+ 333us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:01:48.710710 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:01:48.710940 (+ 230us) client_negotiation.cc:769] Sending connection context
0923 15:01:48.711118 (+ 178us) client_negotiation.cc:240] Negotiation successful
0923 15:01:48.711362 (+ 244us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":195,"thread_start_us":110,"threads_started":1}
I20260923 15:01:48.712352 16856 heartbeater.cc:347] Connected to a master server at 127.13.75.190:34765
I20260923 15:01:48.712486 16856 heartbeater.cc:464] Registering TS with master...
I20260923 15:01:48.712728 16856 heartbeater.cc:511] Master 127.13.75.190:34765 requested a full tablet report, sending...
I20260923 15:01:48.713304 16665 ts_manager.cc:194] Registered new tserver with Master: e6f43c46fa14475493585425be7d6459 (127.13.75.129:33199)
I20260923 15:01:48.713932 16665 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.75.129@KRBTEST.COM'} at 127.13.75.129:44883
WARNING: no policy specified for kudu/127.13.75.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.75.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.75.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.130.keytab.
Entry for principal kudu/127.13.75.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.130.keytab.
I20260923 15:01:48.736048 13614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKC4soy/build/release/bin/kudu
/tmp/dist-test-taskKC4soy/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175691491353-13614-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175691491353-13614-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175691491353-13614-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175691491353-13614-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.75.130:0
--local_ip_for_outbound_sockets=127.13.75.130
--webserver_interface=127.13.75.130
--webserver_port=0
--tserver_master_addrs=127.13.75.190:34765
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.130.keytab
--principal=kudu/127.13.75.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.75.148:46759
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175691491353-13614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175691491353-13614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175691491353-13614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:01:48.819648 16861 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:01:48.819864 16861 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:01:48.819895 16861 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:01:48.821079 16861 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:01:48.821122 16861 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:01:48.821149 16861 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.75.130
I20260923 15:01:48.822656 16861 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.75.148:46759
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175691491353-13614-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175691491353-13614-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.130.keytab
--principal=kudu/127.13.75.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.75.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175691491353-13614-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.75.130
--webserver_port=0
--tserver_master_addrs=127.13.75.190:34765
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16861
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.75.130
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175691491353-13614-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:01:48.822949 16861 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:01:48.823199 16861 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:01:48.828531 16867 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:01:48.828555 16866 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:01:48.828661 16861 server_base.cc:1034] running on GCE node
W20260923 15:01:48.828555 16869 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:01:48.829006 16861 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:01:48.829278 16861 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:01:48.830477 16861 hybrid_clock.cc:648] HybridClock initialized: now 1790175708830463 us; error 29 us; skew 500 ppm
Sep 23 15:01:48 dist-test-slave-cnb0 krb5kdc[16620](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175708, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:01:48.833160 16861 init.cc:377] Logged in from keytab as kudu/127.13.75.130@KRBTEST.COM (short username kudu)
I20260923 15:01:48.833717 16861 webserver.cc:460] Webserver started at http://127.13.75.130:44083/ using document root <none> and password file <none>
I20260923 15:01:48.833988 16861 fs_manager.cc:359] Metadata directory not provided
I20260923 15:01:48.834067 16861 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:01:48.834229 16861 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:01:48.835485 16861 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175691491353-13614-0/minicluster-data/ts-1/data/instance:
uuid: "66711c2a568b4159815352c4a8f8496a"
format_stamp: "Formatted at 2026-09-23 15:01:48 on dist-test-slave-cnb0"
server_key: "3aab037ffca2ce76c68596de947b2922"
server_key_iv: "a48d9f32b047d27a0f8aea2e7f453f0e"
server_key_version: "encryptionkey@0"
I20260923 15:01:48.835879 16861 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175691491353-13614-0/minicluster-data/ts-1/wal/instance:
uuid: "66711c2a568b4159815352c4a8f8496a"
format_stamp: "Formatted at 2026-09-23 15:01:48 on dist-test-slave-cnb0"
server_key: "3aab037ffca2ce76c68596de947b2922"
server_key_iv: "a48d9f32b047d27a0f8aea2e7f453f0e"
server_key_version: "encryptionkey@0"
I20260923 15:01:48.837404 16861 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 15:01:48.838352 16876 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:01:48.838663 16861 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:01:48.838779 16861 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175691491353-13614-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175691491353-13614-0/minicluster-data/ts-1/wal
uuid: "66711c2a568b4159815352c4a8f8496a"
format_stamp: "Formatted at 2026-09-23 15:01:48 on dist-test-slave-cnb0"
server_key: "3aab037ffca2ce76c68596de947b2922"
server_key_iv: "a48d9f32b047d27a0f8aea2e7f453f0e"
server_key_version: "encryptionkey@0"
I20260923 15:01:48.838871 16861 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175691491353-13614-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175691491353-13614-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175691491353-13614-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:01:48.858155 16861 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:01:48.860813 16861 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:01:48.861013 16861 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:01:48.861287 16861 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 15:01:48.861627 16861 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 15:01:48.861688 16861 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 15:01:48.861729 16861 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 15:01:48.861750 16861 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 15:01:48.867986 16861 rpc_server.cc:307] RPC server started. Bound to: 127.13.75.130:41221
I20260923 15:01:48.868026 16989 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.75.130:41221 every 8 connection(s)
I20260923 15:01:48.868432 16861 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175691491353-13614-0/minicluster-data/ts-1/data/info.pb
I20260923 15:01:48.871948 13614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKC4soy/build/release/bin/kudu as pid 16861
I20260923 15:01:48.872076 13614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175691491353-13614-0/minicluster-data/ts-1/wal/instance
I20260923 15:01:48.872305 13614 external_mini_cluster.cc:1428] Setting key 10812955d688e45cecafbcf4be510308
Sep 23 15:01:48 dist-test-slave-cnb0 krb5kdc[16620](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175708, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.130@KRBTEST.COM for kudu/127.13.75.190@KRBTEST.COM
I20260923 15:01:48.879662 16705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:48.869381 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.130:55199 (local address 127.13.75.190:34765)
0923 15:01:48.869427 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0923 15:01:48.869430 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:01:48.869537 (+ 107us) server_negotiation.cc:407] Connection header received
0923 15:01:48.870094 (+ 557us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:01:48.870095 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:01:48.870106 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:01:48.870172 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:01:48.871292 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:48.871888 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:48.873146 (+ 1258us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:48.873463 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:48.873539 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:01:48.876316 (+ 2777us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:01:48.876318 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:01:48.876318 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:01:48.876327 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:01:48.878008 (+ 1681us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:01:48.878520 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:01:48.878521 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:01:48.878522 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:01:48.878599 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:01:48.878912 (+ 313us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:01:48.878913 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:01:48.878914 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:01:48.879051 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:01:48.879144 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0923 15:01:48.879455 (+ 311us) server_negotiation.cc:299] Negotiation successful
0923 15:01:48.879569 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260923 15:01:48.879861 16992 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:48.869198 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:34765 (local address 127.13.75.130:55199)
0923 15:01:48.869392 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0923 15:01:48.869436 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0923 15:01:48.869967 (+ 531us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:01:48.870276 (+ 309us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:01:48.870281 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:01:48.870852 (+ 571us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:01:48.871159 (+ 307us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:01:48.871164 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:01:48.872093 (+ 929us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:01:48.872094 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:01:48.873073 (+ 979us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:01:48.873076 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:01:48.873560 (+ 484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:01:48.873561 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:01:48.873676 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:01:48.874322 (+ 646us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:01:48.874334 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:01:48.876201 (+ 1867us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:01:48.878153 (+ 1952us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:01:48.878163 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:01:48.878165 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:01:48.878425 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:01:48.878680 (+ 255us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:01:48.878681 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:01:48.878681 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:01:48.878808 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:01:48.879133 (+ 325us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:01:48.879141 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:01:48.879310 (+ 169us) client_negotiation.cc:769] Sending connection context
0923 15:01:48.879433 (+ 123us) client_negotiation.cc:240] Negotiation successful
0923 15:01:48.879607 (+ 174us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":104,"threads_started":1}
I20260923 15:01:48.880622 16990 heartbeater.cc:347] Connected to a master server at 127.13.75.190:34765
I20260923 15:01:48.880766 16990 heartbeater.cc:464] Registering TS with master...
I20260923 15:01:48.881001 16990 heartbeater.cc:511] Master 127.13.75.190:34765 requested a full tablet report, sending...
WARNING: no policy specified for kudu/127.13.75.131@KRBTEST.COM; defaulting to no policy
I20260923 15:01:48.881533 16665 ts_manager.cc:194] Registered new tserver with Master: 66711c2a568b4159815352c4a8f8496a (127.13.75.130:41221)
I20260923 15:01:48.881949 16665 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.75.130@KRBTEST.COM'} at 127.13.75.130:55199
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.75.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.75.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.131.keytab.
Entry for principal kudu/127.13.75.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.131.keytab.
I20260923 15:01:48.903126 13614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKC4soy/build/release/bin/kudu
/tmp/dist-test-taskKC4soy/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175691491353-13614-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175691491353-13614-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175691491353-13614-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175691491353-13614-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.75.131:0
--local_ip_for_outbound_sockets=127.13.75.131
--webserver_interface=127.13.75.131
--webserver_port=0
--tserver_master_addrs=127.13.75.190:34765
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.131.keytab
--principal=kudu/127.13.75.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.75.148:46759
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175691491353-13614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175691491353-13614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175691491353-13614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:01:48.987579 16995 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:01:48.987792 16995 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:01:48.987818 16995 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:01:48.989066 16995 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:01:48.989118 16995 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:01:48.989147 16995 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.75.131
I20260923 15:01:48.990648 16995 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.75.148:46759
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175691491353-13614-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175691491353-13614-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.131.keytab
--principal=kudu/127.13.75.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.75.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175691491353-13614-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.75.131
--webserver_port=0
--tserver_master_addrs=127.13.75.190:34765
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.16995
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.75.131
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175691491353-13614-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:01:48.990940 16995 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:01:48.991251 16995 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:01:48.996534 17000 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:01:48.996554 17003 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:01:48.996789 16995 server_base.cc:1034] running on GCE node
W20260923 15:01:48.996543 17001 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:01:48.997066 16995 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:01:48.997344 16995 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:01:48.998588 16995 hybrid_clock.cc:648] HybridClock initialized: now 1790175708998512 us; error 60 us; skew 500 ppm
Sep 23 15:01:48 dist-test-slave-cnb0 krb5kdc[16620](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175708, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:01:49.001577 16995 init.cc:377] Logged in from keytab as kudu/127.13.75.131@KRBTEST.COM (short username kudu)
I20260923 15:01:49.002106 16995 webserver.cc:460] Webserver started at http://127.13.75.131:32819/ using document root <none> and password file <none>
I20260923 15:01:49.002381 16995 fs_manager.cc:359] Metadata directory not provided
I20260923 15:01:49.002431 16995 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:01:49.002566 16995 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:01:49.004067 16995 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175691491353-13614-0/minicluster-data/ts-2/data/instance:
uuid: "b8eb798ef9b54c10afcbf75b82751dcc"
format_stamp: "Formatted at 2026-09-23 15:01:49 on dist-test-slave-cnb0"
server_key: "9118d5fb294dac9690a936f6ec2f3807"
server_key_iv: "7a949b50956250dc0f3fba7bfb0d4366"
server_key_version: "encryptionkey@0"
I20260923 15:01:49.004463 16995 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175691491353-13614-0/minicluster-data/ts-2/wal/instance:
uuid: "b8eb798ef9b54c10afcbf75b82751dcc"
format_stamp: "Formatted at 2026-09-23 15:01:49 on dist-test-slave-cnb0"
server_key: "9118d5fb294dac9690a936f6ec2f3807"
server_key_iv: "7a949b50956250dc0f3fba7bfb0d4366"
server_key_version: "encryptionkey@0"
I20260923 15:01:49.005975 16995 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260923 15:01:49.007133 17010 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:01:49.007547 16995 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:01:49.007668 16995 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175691491353-13614-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175691491353-13614-0/minicluster-data/ts-2/wal
uuid: "b8eb798ef9b54c10afcbf75b82751dcc"
format_stamp: "Formatted at 2026-09-23 15:01:49 on dist-test-slave-cnb0"
server_key: "9118d5fb294dac9690a936f6ec2f3807"
server_key_iv: "7a949b50956250dc0f3fba7bfb0d4366"
server_key_version: "encryptionkey@0"
I20260923 15:01:49.007786 16995 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175691491353-13614-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175691491353-13614-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175691491353-13614-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:01:49.016026 16995 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:01:49.018528 16995 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:01:49.018699 16995 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:01:49.019001 16995 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 15:01:49.019351 16995 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 15:01:49.019387 16995 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 15:01:49.019418 16995 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 15:01:49.019459 16995 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 15:01:49.025640 16995 rpc_server.cc:307] RPC server started. Bound to: 127.13.75.131:46269
I20260923 15:01:49.025677 17123 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.75.131:46269 every 8 connection(s)
I20260923 15:01:49.026031 16995 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175691491353-13614-0/minicluster-data/ts-2/data/info.pb
I20260923 15:01:49.028707 13614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKC4soy/build/release/bin/kudu as pid 16995
I20260923 15:01:49.028827 13614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestUnauthorizedClientKerberosCredentials.1790175691491353-13614-0/minicluster-data/ts-2/wal/instance
I20260923 15:01:49.029016 13614 external_mini_cluster.cc:1428] Setting key bb32ffd1036786bcba831cdcc605122d
Sep 23 15:01:49 dist-test-slave-cnb0 krb5kdc[16620](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175708, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.131@KRBTEST.COM for kudu/127.13.75.190@KRBTEST.COM
I20260923 15:01:49.037017 16705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:49.026847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.131:47743 (local address 127.13.75.190:34765)
0923 15:01:49.026969 (+ 122us) server_negotiation.cc:206] Beginning negotiation
0923 15:01:49.026972 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:01:49.027160 (+ 188us) server_negotiation.cc:407] Connection header received
0923 15:01:49.027831 (+ 671us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:01:49.027837 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:01:49.027852 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:01:49.027904 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:01:49.028882 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:49.029578 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:49.030795 (+ 1217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:49.031139 (+ 344us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:49.031213 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:01:49.033684 (+ 2471us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:01:49.033686 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:01:49.033686 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:01:49.033695 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:01:49.035397 (+ 1702us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:01:49.035940 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:01:49.035941 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:01:49.035941 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:01:49.036004 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:01:49.036305 (+ 301us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:01:49.036306 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:01:49.036306 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:01:49.036474 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:01:49.036534 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0923 15:01:49.036860 (+ 326us) server_negotiation.cc:299] Negotiation successful
0923 15:01:49.036949 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260923 15:01:49.037196 17126 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:49.026750 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:34765 (local address 127.13.75.131:47743)
0923 15:01:49.027028 (+ 278us) negotiation.cc:107] Waiting for socket to connect
0923 15:01:49.027078 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0923 15:01:49.027702 (+ 624us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:01:49.027980 (+ 278us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:01:49.027984 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:01:49.028569 (+ 585us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:01:49.028769 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:01:49.028774 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:01:49.029699 (+ 925us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:01:49.029700 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:01:49.030680 (+ 980us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:01:49.030683 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:01:49.031245 (+ 562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:01:49.031246 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:01:49.031363 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:01:49.031904 (+ 541us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:01:49.031913 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:01:49.033546 (+ 1633us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:01:49.035568 (+ 2022us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:01:49.035575 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:01:49.035577 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:01:49.035841 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:01:49.036103 (+ 262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:01:49.036104 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:01:49.036104 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:01:49.036230 (+ 126us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:01:49.036563 (+ 333us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:01:49.036569 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:01:49.036701 (+ 132us) client_negotiation.cc:769] Sending connection context
0923 15:01:49.036799 (+ 98us) client_negotiation.cc:240] Negotiation successful
0923 15:01:49.036932 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":237,"thread_start_us":142,"threads_started":1}
I20260923 15:01:49.037787 17124 heartbeater.cc:347] Connected to a master server at 127.13.75.190:34765
I20260923 15:01:49.037905 17124 heartbeater.cc:464] Registering TS with master...
I20260923 15:01:49.038146 17124 heartbeater.cc:511] Master 127.13.75.190:34765 requested a full tablet report, sending...
I20260923 15:01:49.038592 16665 ts_manager.cc:194] Registered new tserver with Master: b8eb798ef9b54c10afcbf75b82751dcc (127.13.75.131:46269)
I20260923 15:01:49.039106 16665 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.75.131@KRBTEST.COM'} at 127.13.75.131:47743
I20260923 15:01:49.041172 13614 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 23 15:01:49 dist-test-slave-cnb0 krb5kdc[16620](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175709, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for joe-interloper@KRBTEST.COM:
Sep 23 15:01:49 dist-test-slave-cnb0 krb5kdc[16620](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175709, etypes {rep=17 tkt=17 ses=17}, joe-interloper@KRBTEST.COM for kudu/127.13.75.190@KRBTEST.COM
I20260923 15:01:49.065960 16705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:49.058055 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47362 (local address 127.13.75.190:34765)
0923 15:01:49.058142 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0923 15:01:49.058144 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:01:49.058239 (+ 95us) server_negotiation.cc:407] Connection header received
0923 15:01:49.058296 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:01:49.058297 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:01:49.058315 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:01:49.058369 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:01:49.059176 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:49.059663 (+ 487us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:49.060504 (+ 841us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:49.060710 (+ 206us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:49.060775 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:01:49.063294 (+ 2519us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:01:49.063296 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:01:49.063296 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:01:49.063304 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:01:49.064675 (+ 1371us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:01:49.065025 (+ 350us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:01:49.065026 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:01:49.065026 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:01:49.065081 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:01:49.065373 (+ 292us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:01:49.065374 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:01:49.065375 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:01:49.065546 (+ 171us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:01:49.065638 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0923 15:01:49.065812 (+ 174us) server_negotiation.cc:299] Negotiation successful
0923 15:01:49.065883 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
W20260923 15:01:49.066270 16665 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.ConnectToMaster from {username='joe-interloper', principal='joe-interloper@KRBTEST.COM'} at 127.0.0.1:47362
I20260923 15:01:49.066825 13614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKC4soy/build/release/bin/kudu with pid 16727
I20260923 15:01:49.071879 13614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKC4soy/build/release/bin/kudu with pid 16861
I20260923 15:01:49.076678 13614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKC4soy/build/release/bin/kudu with pid 16995
I20260923 15:01:49.081246 13614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKC4soy/build/release/bin/kudu with pid 16633
2026-09-23T15:01:49Z chronyd exiting
[ OK ] SecurityITest.TestUnauthorizedClientKerberosCredentials (1911 ms)
[ RUN ] SecurityITest.TestAuthorizedSuperuser
Loading random data
Initializing database '/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175691491353-13614-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 15:01:49 dist-test-slave-cnb0 krb5kdc[17136](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 15:01:49 dist-test-slave-cnb0 krb5kdc[17136](info): set up 2 sockets
Sep 23 15:01:49 dist-test-slave-cnb0 krb5kdc[17136](info): commencing operation
krb5kdc: starting...
W20260923 15:01:50.152709 13614 mini_kdc.cc:121] Time spent starting KDC: real 1.051s user 0.003s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175691491353-13614-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175691491353-13614-0/krb5kdc/test-user.keytab.
Sep 23 15:01:50 dist-test-slave-cnb0 krb5kdc[17136](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175710, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T15:01:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T15:01:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.75.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.75.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.75.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.190.keytab.
Entry for principal kudu/127.13.75.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.190.keytab.
I20260923 15:01:50.291296 13614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKC4soy/build/release/bin/kudu
/tmp/dist-test-taskKC4soy/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175691491353-13614-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175691491353-13614-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175691491353-13614-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175691491353-13614-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.75.190:33889
--webserver_interface=127.13.75.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.190.keytab
--principal=kudu/127.13.75.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.75.148:33775
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.75.190:33889
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175691491353-13614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175691491353-13614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175691491353-13614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:01:50.379406 17149 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:01:50.379653 17149 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:01:50.379679 17149 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:01:50.381022 17149 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 15:01:50.381073 17149 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:01:50.381090 17149 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:01:50.381104 17149 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 15:01:50.381119 17149 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 15:01:50.382911 17149 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.75.148:33775
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175691491353-13614-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175691491353-13614-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.75.190:33889
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.190.keytab
--principal=kudu/127.13.75.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.75.190:33889
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175691491353-13614-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.75.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17149
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175691491353-13614-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:01:50.383210 17149 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:01:50.383481 17149 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:01:50.389321 17157 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:01:50.389321 17155 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:01:50.389341 17154 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:01:50.389691 17149 server_base.cc:1034] running on GCE node
I20260923 15:01:50.390051 17149 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:01:50.390413 17149 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:01:50.391635 17149 hybrid_clock.cc:648] HybridClock initialized: now 1790175710391602 us; error 44 us; skew 500 ppm
Sep 23 15:01:50 dist-test-slave-cnb0 krb5kdc[17136](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175710, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:01:50.394459 17149 init.cc:377] Logged in from keytab as kudu/127.13.75.190@KRBTEST.COM (short username kudu)
I20260923 15:01:50.395081 17149 webserver.cc:460] Webserver started at http://127.13.75.190:38293/ using document root <none> and password file <none>
I20260923 15:01:50.395334 17149 fs_manager.cc:359] Metadata directory not provided
I20260923 15:01:50.395411 17149 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:01:50.395572 17149 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:01:50.396689 17149 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175691491353-13614-0/minicluster-data/master-0/data/instance:
uuid: "2b452e791410466d8f2aed6c5f721ae3"
format_stamp: "Formatted at 2026-09-23 15:01:50 on dist-test-slave-cnb0"
server_key: "dc9beada77719390202dddfecccfdb8b"
server_key_iv: "73ae51152a2b816969f55ccbdbd49333"
server_key_version: "encryptionkey@0"
I20260923 15:01:50.397073 17149 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175691491353-13614-0/minicluster-data/master-0/wal/instance:
uuid: "2b452e791410466d8f2aed6c5f721ae3"
format_stamp: "Formatted at 2026-09-23 15:01:50 on dist-test-slave-cnb0"
server_key: "dc9beada77719390202dddfecccfdb8b"
server_key_iv: "73ae51152a2b816969f55ccbdbd49333"
server_key_version: "encryptionkey@0"
I20260923 15:01:50.398576 17149 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 15:01:50.399639 17164 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:01:50.399891 17149 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:01:50.400018 17149 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175691491353-13614-0/minicluster-data/master-0/data,/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175691491353-13614-0/minicluster-data/master-0/wal
uuid: "2b452e791410466d8f2aed6c5f721ae3"
format_stamp: "Formatted at 2026-09-23 15:01:50 on dist-test-slave-cnb0"
server_key: "dc9beada77719390202dddfecccfdb8b"
server_key_iv: "73ae51152a2b816969f55ccbdbd49333"
server_key_version: "encryptionkey@0"
I20260923 15:01:50.400104 17149 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175691491353-13614-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175691491353-13614-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175691491353-13614-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:01:50.428301 17149 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:01:50.430735 17149 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:01:50.430996 17149 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:01:50.435943 17149 rpc_server.cc:307] RPC server started. Bound to: 127.13.75.190:33889
I20260923 15:01:50.435977 17216 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.75.190:33889 every 8 connection(s)
I20260923 15:01:50.436580 17149 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175691491353-13614-0/minicluster-data/master-0/data/info.pb
I20260923 15:01:50.436942 13614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKC4soy/build/release/bin/kudu as pid 17149
I20260923 15:01:50.437063 13614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175691491353-13614-0/minicluster-data/master-0/wal/instance
I20260923 15:01:50.437275 13614 external_mini_cluster.cc:1428] Setting key f6b1c0f05d5bb9ba0a07f7d4e6e5f1a1
I20260923 15:01:50.437500 17217 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:01:50.441258 17217 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2b452e791410466d8f2aed6c5f721ae3: Bootstrap starting.
I20260923 15:01:50.442080 17217 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 2b452e791410466d8f2aed6c5f721ae3: Neither blocks nor log segments found. Creating new log.
I20260923 15:01:50.442425 17217 log.cc:824] T 00000000000000000000000000000000 P 2b452e791410466d8f2aed6c5f721ae3: Log is configured to *not* fsync() on all Append() calls
I20260923 15:01:50.443316 17217 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 2b452e791410466d8f2aed6c5f721ae3: No bootstrap required, opened a new log
Sep 23 15:01:50 dist-test-slave-cnb0 krb5kdc[17136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175710, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.75.190@KRBTEST.COM
I20260923 15:01:50.445851 17217 raft_consensus.cc:348] T 00000000000000000000000000000000 P 2b452e791410466d8f2aed6c5f721ae3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b452e791410466d8f2aed6c5f721ae3" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 33889 } }
I20260923 15:01:50.446066 17217 raft_consensus.cc:374] T 00000000000000000000000000000000 P 2b452e791410466d8f2aed6c5f721ae3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:01:50.446117 17217 raft_consensus.cc:729] T 00000000000000000000000000000000 P 2b452e791410466d8f2aed6c5f721ae3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b452e791410466d8f2aed6c5f721ae3, State: Initialized, Role: FOLLOWER
I20260923 15:01:50.446283 17217 consensus_queue.cc:260] T 00000000000000000000000000000000 P 2b452e791410466d8f2aed6c5f721ae3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b452e791410466d8f2aed6c5f721ae3" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 33889 } }
I20260923 15:01:50.446385 17217 raft_consensus.cc:388] T 00000000000000000000000000000000 P 2b452e791410466d8f2aed6c5f721ae3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 15:01:50.446421 17217 raft_consensus.cc:482] T 00000000000000000000000000000000 P 2b452e791410466d8f2aed6c5f721ae3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 15:01:50.446473 17217 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 2b452e791410466d8f2aed6c5f721ae3 [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:01:50.447216 17217 raft_consensus.cc:504] T 00000000000000000000000000000000 P 2b452e791410466d8f2aed6c5f721ae3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b452e791410466d8f2aed6c5f721ae3" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 33889 } }
I20260923 15:01:50.447345 17217 leader_election.cc:302] T 00000000000000000000000000000000 P 2b452e791410466d8f2aed6c5f721ae3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 2b452e791410466d8f2aed6c5f721ae3; no voters:
I20260923 15:01:50.447561 17217 leader_election.cc:288] T 00000000000000000000000000000000 P 2b452e791410466d8f2aed6c5f721ae3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 15:01:50.447594 17222 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 2b452e791410466d8f2aed6c5f721ae3 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 15:01:50.447700 17222 raft_consensus.cc:686] T 00000000000000000000000000000000 P 2b452e791410466d8f2aed6c5f721ae3 [term 1 LEADER]: Becoming Leader. State: Replica: 2b452e791410466d8f2aed6c5f721ae3, State: Running, Role: LEADER
I20260923 15:01:50.447849 17217 sys_catalog.cc:564] T 00000000000000000000000000000000 P 2b452e791410466d8f2aed6c5f721ae3 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 15:01:50.447847 17222 consensus_queue.cc:237] T 00000000000000000000000000000000 P 2b452e791410466d8f2aed6c5f721ae3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b452e791410466d8f2aed6c5f721ae3" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 33889 } }
I20260923 15:01:50.448213 17222 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b452e791410466d8f2aed6c5f721ae3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 2b452e791410466d8f2aed6c5f721ae3. Latest consensus state: current_term: 1 leader_uuid: "2b452e791410466d8f2aed6c5f721ae3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b452e791410466d8f2aed6c5f721ae3" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 33889 } } }
I20260923 15:01:50.448259 17223 sys_catalog.cc:455] T 00000000000000000000000000000000 P 2b452e791410466d8f2aed6c5f721ae3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "2b452e791410466d8f2aed6c5f721ae3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "2b452e791410466d8f2aed6c5f721ae3" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 33889 } } }
I20260923 15:01:50.448421 17222 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b452e791410466d8f2aed6c5f721ae3 [sys.catalog]: This master's current role is: LEADER
I20260923 15:01:50.448446 17223 sys_catalog.cc:458] T 00000000000000000000000000000000 P 2b452e791410466d8f2aed6c5f721ae3 [sys.catalog]: This master's current role is: LEADER
I20260923 15:01:50.448652 17230 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 15:01:50.449276 17230 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 15:01:50.450831 17220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:50.437882 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34460 (local address 127.13.75.190:33889)
0923 15:01:50.438101 (+ 219us) server_negotiation.cc:206] Beginning negotiation
0923 15:01:50.438114 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0923 15:01:50.438149 (+ 35us) server_negotiation.cc:407] Connection header received
0923 15:01:50.438686 (+ 537us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:01:50.438693 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:01:50.438745 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:01:50.438872 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:01:50.439888 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:50.440937 (+ 1049us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:50.442185 (+ 1248us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:50.442764 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:50.442945 (+ 181us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:01:50.445979 (+ 3034us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:01:50.445984 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:01:50.445984 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:01:50.445998 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:01:50.448434 (+ 2436us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:01:50.448965 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:01:50.448968 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:01:50.448969 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:01:50.449079 (+ 110us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:01:50.449344 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:01:50.449344 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:01:50.449345 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:01:50.449795 (+ 450us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:01:50.450373 (+ 578us) server_negotiation.cc:1035] Waiting for connection context
0923 15:01:50.450455 (+ 82us) server_negotiation.cc:299] Negotiation successful
0923 15:01:50.450613 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":119,"threads_started":1}
I20260923 15:01:50.454448 17230 catalog_manager.cc:1347] Generated new cluster ID: 836e2c96b0004499ab0baf020cab468d
I20260923 15:01:50.454670 17230 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 15:01:50.469777 17230 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 15:01:50.470387 17230 catalog_manager.cc:1504] Loading token signing keys...
I20260923 15:01:50.473270 17230 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 2b452e791410466d8f2aed6c5f721ae3: Generated new TSK 0
I20260923 15:01:50.473470 17230 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.75.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.75.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.75.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.129.keytab.
Entry for principal kudu/127.13.75.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.129.keytab.
I20260923 15:01:50.511960 13614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKC4soy/build/release/bin/kudu
/tmp/dist-test-taskKC4soy/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175691491353-13614-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175691491353-13614-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175691491353-13614-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175691491353-13614-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.75.129:0
--local_ip_for_outbound_sockets=127.13.75.129
--webserver_interface=127.13.75.129
--webserver_port=0
--tserver_master_addrs=127.13.75.190:33889
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.129.keytab
--principal=kudu/127.13.75.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.75.148:33775
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175691491353-13614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175691491353-13614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175691491353-13614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:01:50.629164 17243 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:01:50.629594 17243 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:01:50.629629 17243 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:01:50.630927 17243 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:01:50.630975 17243 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:01:50.631004 17243 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.75.129
I20260923 15:01:50.632660 17243 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.75.148:33775
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175691491353-13614-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175691491353-13614-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.129.keytab
--principal=kudu/127.13.75.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.75.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175691491353-13614-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.75.129
--webserver_port=0
--tserver_master_addrs=127.13.75.190:33889
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17243
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.75.129
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175691491353-13614-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:01:50.632910 17243 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:01:50.633109 17243 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:01:50.638796 17248 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:01:50.638809 17251 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:01:50.638808 17249 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:01:50.639058 17243 server_base.cc:1034] running on GCE node
I20260923 15:01:50.639417 17243 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:01:50.639709 17243 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:01:50.640874 17243 hybrid_clock.cc:648] HybridClock initialized: now 1790175710640845 us; error 40 us; skew 500 ppm
Sep 23 15:01:50 dist-test-slave-cnb0 krb5kdc[17136](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175710, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:01:50.643725 17243 init.cc:377] Logged in from keytab as kudu/127.13.75.129@KRBTEST.COM (short username kudu)
I20260923 15:01:50.644261 17243 webserver.cc:460] Webserver started at http://127.13.75.129:33435/ using document root <none> and password file <none>
I20260923 15:01:50.644564 17243 fs_manager.cc:359] Metadata directory not provided
I20260923 15:01:50.644618 17243 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:01:50.644778 17243 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:01:50.645893 17243 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175691491353-13614-0/minicluster-data/ts-0/data/instance:
uuid: "ee028bee36224555926ef683a108aec8"
format_stamp: "Formatted at 2026-09-23 15:01:50 on dist-test-slave-cnb0"
server_key: "ff99a671b97b5981085829a82ca63e9c"
server_key_iv: "fec2048805b72ef22a121edcd70df053"
server_key_version: "encryptionkey@0"
I20260923 15:01:50.646286 17243 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175691491353-13614-0/minicluster-data/ts-0/wal/instance:
uuid: "ee028bee36224555926ef683a108aec8"
format_stamp: "Formatted at 2026-09-23 15:01:50 on dist-test-slave-cnb0"
server_key: "ff99a671b97b5981085829a82ca63e9c"
server_key_iv: "fec2048805b72ef22a121edcd70df053"
server_key_version: "encryptionkey@0"
I20260923 15:01:50.647969 17243 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 15:01:50.649032 17258 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:01:50.649214 17243 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:01:50.649300 17243 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175691491353-13614-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175691491353-13614-0/minicluster-data/ts-0/wal
uuid: "ee028bee36224555926ef683a108aec8"
format_stamp: "Formatted at 2026-09-23 15:01:50 on dist-test-slave-cnb0"
server_key: "ff99a671b97b5981085829a82ca63e9c"
server_key_iv: "fec2048805b72ef22a121edcd70df053"
server_key_version: "encryptionkey@0"
I20260923 15:01:50.649358 17243 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175691491353-13614-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175691491353-13614-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175691491353-13614-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:01:50.666766 17243 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:01:50.669221 17243 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:01:50.669358 17243 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:01:50.669588 17243 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 15:01:50.669850 17243 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 15:01:50.669881 17243 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 15:01:50.669905 17243 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 15:01:50.669919 17243 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 15:01:50.676966 17243 rpc_server.cc:307] RPC server started. Bound to: 127.13.75.129:40167
I20260923 15:01:50.677049 17371 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.75.129:40167 every 8 connection(s)
I20260923 15:01:50.677394 17243 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175691491353-13614-0/minicluster-data/ts-0/data/info.pb
I20260923 15:01:50.678836 13614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKC4soy/build/release/bin/kudu as pid 17243
I20260923 15:01:50.678998 13614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175691491353-13614-0/minicluster-data/ts-0/wal/instance
I20260923 15:01:50.679203 13614 external_mini_cluster.cc:1428] Setting key d5b38c5b935173ab22720382068c14b6
Sep 23 15:01:50 dist-test-slave-cnb0 krb5kdc[17136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175710, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.129@KRBTEST.COM for kudu/127.13.75.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.13.75.130@KRBTEST.COM; defaulting to no policy
I20260923 15:01:50.690271 17220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:50.678254 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.129:41237 (local address 127.13.75.190:33889)
0923 15:01:50.678302 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0923 15:01:50.678304 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:01:50.678580 (+ 276us) server_negotiation.cc:407] Connection header received
0923 15:01:50.679957 (+ 1377us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:01:50.679959 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:01:50.679973 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:01:50.680041 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:01:50.680804 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:50.681587 (+ 783us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:50.683250 (+ 1663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:50.683642 (+ 392us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:50.683729 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:01:50.686434 (+ 2705us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:01:50.686436 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:01:50.686437 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:01:50.686446 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:01:50.688191 (+ 1745us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:01:50.688985 (+ 794us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:01:50.688987 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:01:50.688987 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:01:50.689063 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:01:50.689397 (+ 334us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:01:50.689398 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:01:50.689399 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:01:50.689539 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:01:50.689634 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0923 15:01:50.690054 (+ 420us) server_negotiation.cc:299] Negotiation successful
0923 15:01:50.690168 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260923 15:01:50.690502 17374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:50.678210 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:33889 (local address 127.13.75.129:41237)
0923 15:01:50.678394 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0923 15:01:50.678438 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0923 15:01:50.679795 (+ 1357us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:01:50.680157 (+ 362us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:01:50.680162 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:01:50.680435 (+ 273us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:01:50.680680 (+ 245us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:01:50.680685 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:01:50.681742 (+ 1057us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:01:50.681744 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:01:50.683144 (+ 1400us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:01:50.683147 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:01:50.683755 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:01:50.683756 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:01:50.683875 (+ 119us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:01:50.684588 (+ 713us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:01:50.684597 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:01:50.686302 (+ 1705us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:01:50.688321 (+ 2019us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:01:50.688331 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:01:50.688334 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:01:50.688856 (+ 522us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:01:50.689155 (+ 299us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:01:50.689156 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:01:50.689156 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:01:50.689302 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:01:50.689715 (+ 413us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:01:50.689728 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:01:50.689870 (+ 142us) client_negotiation.cc:769] Sending connection context
0923 15:01:50.690005 (+ 135us) client_negotiation.cc:240] Negotiation successful
0923 15:01:50.690196 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":104,"threads_started":1}
I20260923 15:01:50.691274 17372 heartbeater.cc:347] Connected to a master server at 127.13.75.190:33889
I20260923 15:01:50.691388 17372 heartbeater.cc:464] Registering TS with master...
I20260923 15:01:50.691656 17372 heartbeater.cc:511] Master 127.13.75.190:33889 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.75.130@KRBTEST.COM" created.
I20260923 15:01:50.692356 17181 ts_manager.cc:194] Registered new tserver with Master: ee028bee36224555926ef683a108aec8 (127.13.75.129:40167)
I20260923 15:01:50.693149 17181 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.75.129@KRBTEST.COM'} at 127.13.75.129:41237
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.75.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.130.keytab.
Entry for principal kudu/127.13.75.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.130.keytab.
I20260923 15:01:50.715844 13614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKC4soy/build/release/bin/kudu
/tmp/dist-test-taskKC4soy/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175691491353-13614-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175691491353-13614-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175691491353-13614-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175691491353-13614-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.75.130:0
--local_ip_for_outbound_sockets=127.13.75.130
--webserver_interface=127.13.75.130
--webserver_port=0
--tserver_master_addrs=127.13.75.190:33889
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.130.keytab
--principal=kudu/127.13.75.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.75.148:33775
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175691491353-13614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175691491353-13614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175691491353-13614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:01:50.804481 17377 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:01:50.804754 17377 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:01:50.804800 17377 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:01:50.806052 17377 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:01:50.806134 17377 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:01:50.806180 17377 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.75.130
I20260923 15:01:50.807895 17377 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.75.148:33775
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175691491353-13614-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175691491353-13614-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.130.keytab
--principal=kudu/127.13.75.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.75.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175691491353-13614-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.75.130
--webserver_port=0
--tserver_master_addrs=127.13.75.190:33889
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17377
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.75.130
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175691491353-13614-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:01:50.808236 17377 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:01:50.808501 17377 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:01:50.814234 17385 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:01:50.814256 17383 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:01:50.814234 17382 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:01:50.814985 17377 server_base.cc:1034] running on GCE node
I20260923 15:01:50.815294 17377 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:01:50.815603 17377 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:01:50.816819 17377 hybrid_clock.cc:648] HybridClock initialized: now 1790175710816784 us; error 54 us; skew 500 ppm
Sep 23 15:01:50 dist-test-slave-cnb0 krb5kdc[17136](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175710, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:01:50.819661 17377 init.cc:377] Logged in from keytab as kudu/127.13.75.130@KRBTEST.COM (short username kudu)
I20260923 15:01:50.820219 17377 webserver.cc:460] Webserver started at http://127.13.75.130:42671/ using document root <none> and password file <none>
I20260923 15:01:50.820492 17377 fs_manager.cc:359] Metadata directory not provided
I20260923 15:01:50.820542 17377 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:01:50.820681 17377 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:01:50.821763 17377 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175691491353-13614-0/minicluster-data/ts-1/data/instance:
uuid: "a8ea7087621349bcb316ad380555cbf3"
format_stamp: "Formatted at 2026-09-23 15:01:50 on dist-test-slave-cnb0"
server_key: "207f2c03faa59f893dba11ec42355afe"
server_key_iv: "34330717e6452ee297f59e41cf60a65c"
server_key_version: "encryptionkey@0"
I20260923 15:01:50.822142 17377 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175691491353-13614-0/minicluster-data/ts-1/wal/instance:
uuid: "a8ea7087621349bcb316ad380555cbf3"
format_stamp: "Formatted at 2026-09-23 15:01:50 on dist-test-slave-cnb0"
server_key: "207f2c03faa59f893dba11ec42355afe"
server_key_iv: "34330717e6452ee297f59e41cf60a65c"
server_key_version: "encryptionkey@0"
I20260923 15:01:50.823678 17377 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 15:01:50.824640 17392 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:01:50.824880 17377 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:01:50.824992 17377 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175691491353-13614-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175691491353-13614-0/minicluster-data/ts-1/wal
uuid: "a8ea7087621349bcb316ad380555cbf3"
format_stamp: "Formatted at 2026-09-23 15:01:50 on dist-test-slave-cnb0"
server_key: "207f2c03faa59f893dba11ec42355afe"
server_key_iv: "34330717e6452ee297f59e41cf60a65c"
server_key_version: "encryptionkey@0"
I20260923 15:01:50.825078 17377 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175691491353-13614-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175691491353-13614-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175691491353-13614-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:01:50.853523 17377 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:01:50.856145 17377 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:01:50.856324 17377 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:01:50.856693 17377 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 15:01:50.857052 17377 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 15:01:50.857092 17377 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 15:01:50.857117 17377 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 15:01:50.857131 17377 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 15:01:50.863740 17377 rpc_server.cc:307] RPC server started. Bound to: 127.13.75.130:37153
I20260923 15:01:50.863817 17505 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.75.130:37153 every 8 connection(s)
I20260923 15:01:50.864163 17377 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175691491353-13614-0/minicluster-data/ts-1/data/info.pb
Sep 23 15:01:50 dist-test-slave-cnb0 krb5kdc[17136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175710, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.130@KRBTEST.COM for kudu/127.13.75.190@KRBTEST.COM
I20260923 15:01:50.872007 13614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKC4soy/build/release/bin/kudu as pid 17377
I20260923 15:01:50.872136 13614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175691491353-13614-0/minicluster-data/ts-1/wal/instance
I20260923 15:01:50.872362 13614 external_mini_cluster.cc:1428] Setting key 0a550629d08fb5a317903bc6681f70d4
I20260923 15:01:50.874439 17220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:50.865036 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.130:49107 (local address 127.13.75.190:33889)
0923 15:01:50.865127 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0923 15:01:50.865129 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:01:50.865278 (+ 149us) server_negotiation.cc:407] Connection header received
0923 15:01:50.866242 (+ 964us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:01:50.866243 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:01:50.866256 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:01:50.866387 (+ 131us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:01:50.867128 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:50.867666 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:50.868708 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:50.868899 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:50.868958 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:01:50.871185 (+ 2227us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:01:50.871186 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:01:50.871186 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:01:50.871194 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:01:50.872919 (+ 1725us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:01:50.873389 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:01:50.873391 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:01:50.873391 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:01:50.873492 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:01:50.873734 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:01:50.873735 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:01:50.873736 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:01:50.873885 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:01:50.873955 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0923 15:01:50.874202 (+ 247us) server_negotiation.cc:299] Negotiation successful
0923 15:01:50.874291 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":58}
I20260923 15:01:50.874691 17508 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:50.864973 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:33889 (local address 127.13.75.130:49107)
0923 15:01:50.865137 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0923 15:01:50.865183 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0923 15:01:50.866112 (+ 929us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:01:50.866456 (+ 344us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:01:50.866459 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:01:50.866715 (+ 256us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:01:50.867019 (+ 304us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:01:50.867025 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:01:50.867778 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:01:50.867779 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:01:50.868615 (+ 836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:01:50.868617 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:01:50.868989 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:01:50.868990 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:01:50.869097 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:01:50.869578 (+ 481us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:01:50.869587 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:01:50.871077 (+ 1490us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:01:50.873039 (+ 1962us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:01:50.873047 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:01:50.873050 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:01:50.873311 (+ 261us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:01:50.873567 (+ 256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:01:50.873567 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:01:50.873568 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:01:50.873668 (+ 100us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:01:50.873954 (+ 286us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:01:50.873958 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:01:50.874073 (+ 115us) client_negotiation.cc:769] Sending connection context
0923 15:01:50.874156 (+ 83us) client_negotiation.cc:240] Negotiation successful
0923 15:01:50.874277 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":103,"thread_start_us":96,"threads_started":1}
I20260923 15:01:50.875507 17506 heartbeater.cc:347] Connected to a master server at 127.13.75.190:33889
I20260923 15:01:50.875689 17506 heartbeater.cc:464] Registering TS with master...
I20260923 15:01:50.875908 17506 heartbeater.cc:511] Master 127.13.75.190:33889 requested a full tablet report, sending...
I20260923 15:01:50.876434 17181 ts_manager.cc:194] Registered new tserver with Master: a8ea7087621349bcb316ad380555cbf3 (127.13.75.130:37153)
I20260923 15:01:50.876864 17181 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.75.130@KRBTEST.COM'} at 127.13.75.130:49107
WARNING: no policy specified for kudu/127.13.75.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.75.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.75.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.131.keytab.
Entry for principal kudu/127.13.75.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.131.keytab.
I20260923 15:01:50.911263 13614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKC4soy/build/release/bin/kudu
/tmp/dist-test-taskKC4soy/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175691491353-13614-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175691491353-13614-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175691491353-13614-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175691491353-13614-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.75.131:0
--local_ip_for_outbound_sockets=127.13.75.131
--webserver_interface=127.13.75.131
--webserver_port=0
--tserver_master_addrs=127.13.75.190:33889
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.131.keytab
--principal=kudu/127.13.75.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.75.148:33775
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175691491353-13614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175691491353-13614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175691491353-13614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:01:50.999735 17511 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:01:50.999982 17511 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:01:51.000010 17511 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:01:51.001194 17511 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:01:51.001237 17511 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:01:51.001267 17511 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.75.131
I20260923 15:01:51.003008 17511 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.75.148:33775
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175691491353-13614-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175691491353-13614-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.131.keytab
--principal=kudu/127.13.75.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.75.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175691491353-13614-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.75.131
--webserver_port=0
--tserver_master_addrs=127.13.75.190:33889
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17511
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.75.131
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175691491353-13614-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:01:51.003244 17511 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:01:51.003449 17511 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:01:51.009341 17516 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:01:51.009352 17519 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:01:51.009339 17517 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:01:51.009733 17511 server_base.cc:1034] running on GCE node
I20260923 15:01:51.009931 17511 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:01:51.010234 17511 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:01:51.011448 17511 hybrid_clock.cc:648] HybridClock initialized: now 1790175711011404 us; error 59 us; skew 500 ppm
Sep 23 15:01:51 dist-test-slave-cnb0 krb5kdc[17136](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175711, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:01:51.014225 17511 init.cc:377] Logged in from keytab as kudu/127.13.75.131@KRBTEST.COM (short username kudu)
I20260923 15:01:51.014708 17511 webserver.cc:460] Webserver started at http://127.13.75.131:37257/ using document root <none> and password file <none>
I20260923 15:01:51.015034 17511 fs_manager.cc:359] Metadata directory not provided
I20260923 15:01:51.015110 17511 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:01:51.015259 17511 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:01:51.016340 17511 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175691491353-13614-0/minicluster-data/ts-2/data/instance:
uuid: "d7c358e0a35d477abbc4557a465d3a92"
format_stamp: "Formatted at 2026-09-23 15:01:51 on dist-test-slave-cnb0"
server_key: "7f274f45b73d4f62c3dcf25aaeef88bb"
server_key_iv: "13dd3612295831ccbaaba48a27c34b1c"
server_key_version: "encryptionkey@0"
I20260923 15:01:51.016800 17511 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175691491353-13614-0/minicluster-data/ts-2/wal/instance:
uuid: "d7c358e0a35d477abbc4557a465d3a92"
format_stamp: "Formatted at 2026-09-23 15:01:51 on dist-test-slave-cnb0"
server_key: "7f274f45b73d4f62c3dcf25aaeef88bb"
server_key_iv: "13dd3612295831ccbaaba48a27c34b1c"
server_key_version: "encryptionkey@0"
I20260923 15:01:51.018337 17511 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 15:01:51.019236 17526 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:01:51.019557 17511 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260923 15:01:51.019676 17511 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175691491353-13614-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175691491353-13614-0/minicluster-data/ts-2/wal
uuid: "d7c358e0a35d477abbc4557a465d3a92"
format_stamp: "Formatted at 2026-09-23 15:01:51 on dist-test-slave-cnb0"
server_key: "7f274f45b73d4f62c3dcf25aaeef88bb"
server_key_iv: "13dd3612295831ccbaaba48a27c34b1c"
server_key_version: "encryptionkey@0"
I20260923 15:01:51.019769 17511 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175691491353-13614-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175691491353-13614-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175691491353-13614-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:01:51.029222 17511 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:01:51.031697 17511 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:01:51.031874 17511 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:01:51.032127 17511 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 15:01:51.032483 17511 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 15:01:51.032544 17511 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 15:01:51.032585 17511 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 15:01:51.032606 17511 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 15:01:51.038627 17511 rpc_server.cc:307] RPC server started. Bound to: 127.13.75.131:33191
I20260923 15:01:51.038712 17639 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.75.131:33191 every 8 connection(s)
I20260923 15:01:51.039076 17511 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175691491353-13614-0/minicluster-data/ts-2/data/info.pb
Sep 23 15:01:51 dist-test-slave-cnb0 krb5kdc[17136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175711, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.131@KRBTEST.COM for kudu/127.13.75.190@KRBTEST.COM
I20260923 15:01:51.047087 13614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKC4soy/build/release/bin/kudu as pid 17511
I20260923 15:01:51.047192 13614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestAuthorizedSuperuser.1790175691491353-13614-0/minicluster-data/ts-2/wal/instance
I20260923 15:01:51.047398 13614 external_mini_cluster.cc:1428] Setting key 550d656f9d176548e9f6d87084c5a291
I20260923 15:01:51.049929 17220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:51.039991 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.131:48725 (local address 127.13.75.190:33889)
0923 15:01:51.040097 (+ 106us) server_negotiation.cc:206] Beginning negotiation
0923 15:01:51.040100 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:01:51.040266 (+ 166us) server_negotiation.cc:407] Connection header received
0923 15:01:51.041262 (+ 996us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:01:51.041264 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:01:51.041277 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:01:51.041342 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:01:51.042068 (+ 726us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:51.042580 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:51.043687 (+ 1107us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:51.043890 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:51.043991 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:01:51.046353 (+ 2362us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:01:51.046355 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:01:51.046355 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:01:51.046363 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:01:51.048422 (+ 2059us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:01:51.048846 (+ 424us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:01:51.048848 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:01:51.048848 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:01:51.048906 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:01:51.049239 (+ 333us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:01:51.049240 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:01:51.049240 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:01:51.049369 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:01:51.049449 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0923 15:01:51.049752 (+ 303us) server_negotiation.cc:299] Negotiation successful
0923 15:01:51.049840 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260923 15:01:51.050130 17642 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:51.039880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:33889 (local address 127.13.75.131:48725)
0923 15:01:51.040114 (+ 234us) negotiation.cc:107] Waiting for socket to connect
0923 15:01:51.040167 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0923 15:01:51.041134 (+ 967us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:01:51.041413 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:01:51.041418 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:01:51.041692 (+ 274us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:01:51.041889 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:01:51.041893 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:01:51.042684 (+ 791us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:01:51.042686 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:01:51.043583 (+ 897us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:01:51.043586 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:01:51.044003 (+ 417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:01:51.044004 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:01:51.044140 (+ 136us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:01:51.044723 (+ 583us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:01:51.044732 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:01:51.046256 (+ 1524us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:01:51.048495 (+ 2239us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:01:51.048503 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:01:51.048505 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:01:51.048735 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:01:51.049012 (+ 277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:01:51.049013 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:01:51.049013 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:01:51.049165 (+ 152us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:01:51.049440 (+ 275us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:01:51.049447 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:01:51.049603 (+ 156us) client_negotiation.cc:769] Sending connection context
0923 15:01:51.049714 (+ 111us) client_negotiation.cc:240] Negotiation successful
0923 15:01:51.049885 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":190,"thread_start_us":112,"threads_started":1}
I20260923 15:01:51.050631 17640 heartbeater.cc:347] Connected to a master server at 127.13.75.190:33889
I20260923 15:01:51.050737 17640 heartbeater.cc:464] Registering TS with master...
I20260923 15:01:51.050963 17640 heartbeater.cc:511] Master 127.13.75.190:33889 requested a full tablet report, sending...
I20260923 15:01:51.051438 17181 ts_manager.cc:194] Registered new tserver with Master: d7c358e0a35d477abbc4557a465d3a92 (127.13.75.131:33191)
I20260923 15:01:51.051960 17181 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.75.131@KRBTEST.COM'} at 127.13.75.131:48725
I20260923 15:01:51.060173 13614 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Sep 23 15:01:51 dist-test-slave-cnb0 krb5kdc[17136](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175711, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 23 15:01:51 dist-test-slave-cnb0 krb5kdc[17136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175711, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.75.129@KRBTEST.COM
I20260923 15:01:51.085079 17647 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:51.076693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36062 (local address 127.13.75.129:40167)
0923 15:01:51.076858 (+ 165us) server_negotiation.cc:206] Beginning negotiation
0923 15:01:51.076863 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 15:01:51.076872 (+ 9us) server_negotiation.cc:407] Connection header received
0923 15:01:51.076898 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:01:51.076899 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:01:51.076914 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:01:51.077015 (+ 101us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:01:51.077627 (+ 612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:51.078351 (+ 724us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:51.079228 (+ 877us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:51.079562 (+ 334us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:51.079661 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:01:51.082061 (+ 2400us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:01:51.082068 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:01:51.082069 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:01:51.082076 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:01:51.083767 (+ 1691us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:01:51.084122 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:01:51.084125 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:01:51.084125 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:01:51.084191 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:01:51.084420 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:01:51.084421 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:01:51.084421 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:01:51.084595 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:01:51.084713 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0923 15:01:51.084906 (+ 193us) server_negotiation.cc:299] Negotiation successful
0923 15:01:51.084993 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":70,"threads_started":1}
Sep 23 15:01:51 dist-test-slave-cnb0 krb5kdc[17136](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175711, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.75.190@KRBTEST.COM
I20260923 15:01:51.094036 17220 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:51.086734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34466 (local address 127.13.75.190:33889)
0923 15:01:51.086807 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0923 15:01:51.086809 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:01:51.086841 (+ 32us) server_negotiation.cc:407] Connection header received
0923 15:01:51.086952 (+ 111us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:01:51.086953 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:01:51.086964 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:01:51.087026 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:01:51.087520 (+ 494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:51.088049 (+ 529us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:51.088774 (+ 725us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:51.089000 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:51.089063 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:01:51.091369 (+ 2306us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:01:51.091371 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:01:51.091371 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:01:51.091378 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:01:51.092990 (+ 1612us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:01:51.093335 (+ 345us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:01:51.093336 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:01:51.093337 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:01:51.093389 (+ 52us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:01:51.093578 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:01:51.093578 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:01:51.093579 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:01:51.093731 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:01:51.093810 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0923 15:01:51.093907 (+ 97us) server_negotiation.cc:299] Negotiation successful
0923 15:01:51.093968 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
W20260923 15:01:51.094304 17181 server_base.cc:1116] Unauthorized access attempt to method kudu.master.MasterService.TSHeartbeat from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34466
I20260923 15:01:51.094738 13614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKC4soy/build/release/bin/kudu with pid 17243
I20260923 15:01:51.099592 13614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKC4soy/build/release/bin/kudu with pid 17377
I20260923 15:01:51.104744 13614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKC4soy/build/release/bin/kudu with pid 17511
I20260923 15:01:51.109370 13614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKC4soy/build/release/bin/kudu with pid 17149
2026-09-23T15:01:51Z chronyd exiting
[ OK ] SecurityITest.TestAuthorizedSuperuser (2028 ms)
[ RUN ] SecurityITest.TestDisableWebUI
Loading random data
Initializing database '/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175691491353-13614-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 15:01:51 dist-test-slave-cnb0 krb5kdc[17652](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 15:01:51 dist-test-slave-cnb0 krb5kdc[17652](info): set up 2 sockets
Sep 23 15:01:51 dist-test-slave-cnb0 krb5kdc[17652](info): commencing operation
krb5kdc: starting...
W20260923 15:01:52.216946 13614 mini_kdc.cc:121] Time spent starting KDC: real 1.086s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175691491353-13614-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175691491353-13614-0/krb5kdc/test-user.keytab.
Sep 23 15:01:52 dist-test-slave-cnb0 krb5kdc[17652](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175712, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T15:01:52Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T15:01:52Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.75.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.75.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.75.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.190.keytab.
Entry for principal kudu/127.13.75.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.190.keytab.
I20260923 15:01:52.347126 13614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKC4soy/build/release/bin/kudu
/tmp/dist-test-taskKC4soy/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175691491353-13614-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175691491353-13614-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175691491353-13614-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175691491353-13614-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.75.190:42209
--webserver_interface=127.13.75.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.190.keytab
--principal=kudu/127.13.75.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.75.148:43249
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.75.190:42209
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175691491353-13614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175691491353-13614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175691491353-13614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:01:52.429698 17665 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:01:52.429924 17665 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:01:52.429948 17665 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:01:52.431123 17665 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 15:01:52.431160 17665 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:01:52.431174 17665 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:01:52.431187 17665 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 15:01:52.431200 17665 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 15:01:52.432787 17665 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.75.148:43249
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175691491353-13614-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175691491353-13614-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.75.190:42209
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.190.keytab
--principal=kudu/127.13.75.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.75.190:42209
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175691491353-13614-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.75.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17665
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175691491353-13614-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:01:52.432993 17665 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:01:52.433177 17665 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260923 15:01:52.438269 17665 server_base.cc:1034] running on GCE node
W20260923 15:01:52.438249 17673 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:01:52.438196 17670 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:01:52.438196 17671 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:01:52.438704 17665 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:01:52.439049 17665 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:01:52.440218 17665 hybrid_clock.cc:648] HybridClock initialized: now 1790175712440204 us; error 32 us; skew 500 ppm
Sep 23 15:01:52 dist-test-slave-cnb0 krb5kdc[17652](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175712, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:01:52.447384 17665 init.cc:377] Logged in from keytab as kudu/127.13.75.190@KRBTEST.COM (short username kudu)
I20260923 15:01:52.447799 17665 fs_manager.cc:359] Metadata directory not provided
I20260923 15:01:52.447852 17665 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:01:52.448017 17665 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:01:52.449146 17665 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175691491353-13614-0/minicluster-data/master-0/data/instance:
uuid: "4bb808cd65f94e19a9433ece9eb1281a"
format_stamp: "Formatted at 2026-09-23 15:01:52 on dist-test-slave-cnb0"
server_key: "edda09f20d31a01cf8d2824d0568a6dd"
server_key_iv: "a70b2809b47cd3d0f6284340d41fc649"
server_key_version: "encryptionkey@0"
I20260923 15:01:52.449527 17665 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175691491353-13614-0/minicluster-data/master-0/wal/instance:
uuid: "4bb808cd65f94e19a9433ece9eb1281a"
format_stamp: "Formatted at 2026-09-23 15:01:52 on dist-test-slave-cnb0"
server_key: "edda09f20d31a01cf8d2824d0568a6dd"
server_key_iv: "a70b2809b47cd3d0f6284340d41fc649"
server_key_version: "encryptionkey@0"
I20260923 15:01:52.451004 17665 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 15:01:52.452090 17679 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:01:52.452410 17665 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:01:52.452505 17665 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175691491353-13614-0/minicluster-data/master-0/data,/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175691491353-13614-0/minicluster-data/master-0/wal
uuid: "4bb808cd65f94e19a9433ece9eb1281a"
format_stamp: "Formatted at 2026-09-23 15:01:52 on dist-test-slave-cnb0"
server_key: "edda09f20d31a01cf8d2824d0568a6dd"
server_key_iv: "a70b2809b47cd3d0f6284340d41fc649"
server_key_version: "encryptionkey@0"
I20260923 15:01:52.452565 17665 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175691491353-13614-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175691491353-13614-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175691491353-13614-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:01:52.468258 17665 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:01:52.470826 17665 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:01:52.471031 17665 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:01:52.475059 17665 rpc_server.cc:307] RPC server started. Bound to: 127.13.75.190:42209
I20260923 15:01:52.475154 17731 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.75.190:42209 every 8 connection(s)
I20260923 15:01:52.475443 17665 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175691491353-13614-0/minicluster-data/master-0/data/info.pb
I20260923 15:01:52.476478 17732 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:01:52.478772 17732 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4bb808cd65f94e19a9433ece9eb1281a: Bootstrap starting.
I20260923 15:01:52.479528 17732 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4bb808cd65f94e19a9433ece9eb1281a: Neither blocks nor log segments found. Creating new log.
I20260923 15:01:52.479781 17732 log.cc:824] T 00000000000000000000000000000000 P 4bb808cd65f94e19a9433ece9eb1281a: Log is configured to *not* fsync() on all Append() calls
I20260923 15:01:52.480451 17732 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4bb808cd65f94e19a9433ece9eb1281a: No bootstrap required, opened a new log
I20260923 15:01:52.482009 17732 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4bb808cd65f94e19a9433ece9eb1281a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4bb808cd65f94e19a9433ece9eb1281a" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 42209 } }
I20260923 15:01:52.482149 17732 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4bb808cd65f94e19a9433ece9eb1281a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:01:52.482172 17732 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4bb808cd65f94e19a9433ece9eb1281a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4bb808cd65f94e19a9433ece9eb1281a, State: Initialized, Role: FOLLOWER
I20260923 15:01:52.482318 17732 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4bb808cd65f94e19a9433ece9eb1281a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4bb808cd65f94e19a9433ece9eb1281a" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 42209 } }
I20260923 15:01:52.482402 17732 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4bb808cd65f94e19a9433ece9eb1281a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 15:01:52.482426 17732 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4bb808cd65f94e19a9433ece9eb1281a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 15:01:52.482501 17732 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4bb808cd65f94e19a9433ece9eb1281a [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:01:52.482599 13614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKC4soy/build/release/bin/kudu as pid 17665
I20260923 15:01:52.482731 13614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175691491353-13614-0/minicluster-data/master-0/wal/instance
I20260923 15:01:52.482996 13614 external_mini_cluster.cc:1428] Setting key c7f023d8271b8a36d2f8a8672f428cf7
I20260923 15:01:52.483727 17732 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4bb808cd65f94e19a9433ece9eb1281a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4bb808cd65f94e19a9433ece9eb1281a" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 42209 } }
I20260923 15:01:52.483870 17732 leader_election.cc:302] T 00000000000000000000000000000000 P 4bb808cd65f94e19a9433ece9eb1281a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4bb808cd65f94e19a9433ece9eb1281a; no voters:
I20260923 15:01:52.484042 17732 leader_election.cc:288] T 00000000000000000000000000000000 P 4bb808cd65f94e19a9433ece9eb1281a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 15:01:52.484127 17737 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4bb808cd65f94e19a9433ece9eb1281a [term 1 FOLLOWER]: Leader election won for term 1
I20260923 15:01:52.484269 17737 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4bb808cd65f94e19a9433ece9eb1281a [term 1 LEADER]: Becoming Leader. State: Replica: 4bb808cd65f94e19a9433ece9eb1281a, State: Running, Role: LEADER
I20260923 15:01:52.484289 17732 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4bb808cd65f94e19a9433ece9eb1281a [sys.catalog]: configured and running, proceeding with master startup.
I20260923 15:01:52.484367 17737 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4bb808cd65f94e19a9433ece9eb1281a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4bb808cd65f94e19a9433ece9eb1281a" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 42209 } }
I20260923 15:01:52.484671 17737 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4bb808cd65f94e19a9433ece9eb1281a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4bb808cd65f94e19a9433ece9eb1281a. Latest consensus state: current_term: 1 leader_uuid: "4bb808cd65f94e19a9433ece9eb1281a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4bb808cd65f94e19a9433ece9eb1281a" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 42209 } } }
I20260923 15:01:52.484759 17737 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4bb808cd65f94e19a9433ece9eb1281a [sys.catalog]: This master's current role is: LEADER
I20260923 15:01:52.484683 17739 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4bb808cd65f94e19a9433ece9eb1281a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4bb808cd65f94e19a9433ece9eb1281a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4bb808cd65f94e19a9433ece9eb1281a" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 42209 } } }
I20260923 15:01:52.484905 17739 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4bb808cd65f94e19a9433ece9eb1281a [sys.catalog]: This master's current role is: LEADER
I20260923 15:01:52.484969 17744 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 15:01:52.485507 17744 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 15:01:52.487113 17744 catalog_manager.cc:1347] Generated new cluster ID: 9ef6813dfac14dd8a52f803de9fd7705
I20260923 15:01:52.487165 17744 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 23 15:01:52 dist-test-slave-cnb0 krb5kdc[17652](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175712, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.75.190@KRBTEST.COM
I20260923 15:01:52.495330 17736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:52.483629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44290 (local address 127.13.75.190:42209)
0923 15:01:52.483849 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0923 15:01:52.483859 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0923 15:01:52.483899 (+ 40us) server_negotiation.cc:407] Connection header received
0923 15:01:52.484026 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:01:52.484034 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:01:52.484095 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:01:52.484246 (+ 151us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:01:52.485218 (+ 972us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:52.486709 (+ 1491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:52.487770 (+ 1061us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:52.488369 (+ 599us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:52.488471 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:01:52.491375 (+ 2904us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:01:52.491380 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:01:52.491381 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:01:52.491394 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:01:52.493575 (+ 2181us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:01:52.493982 (+ 407us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:01:52.493985 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:01:52.493986 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:01:52.494105 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:01:52.494319 (+ 214us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:01:52.494319 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:01:52.494320 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:01:52.494664 (+ 344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:01:52.494837 (+ 173us) server_negotiation.cc:1035] Waiting for connection context
0923 15:01:52.494938 (+ 101us) server_negotiation.cc:299] Negotiation successful
0923 15:01:52.495090 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":99,"threads_started":1}
I20260923 15:01:52.504570 17744 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 15:01:52.505261 17744 catalog_manager.cc:1504] Loading token signing keys...
I20260923 15:01:52.516520 17744 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4bb808cd65f94e19a9433ece9eb1281a: Generated new TSK 0
I20260923 15:01:52.516755 17744 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.75.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.75.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.75.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.129.keytab.
Entry for principal kudu/127.13.75.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.129.keytab.
I20260923 15:01:52.555662 13614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKC4soy/build/release/bin/kudu
/tmp/dist-test-taskKC4soy/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175691491353-13614-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175691491353-13614-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175691491353-13614-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175691491353-13614-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.75.129:0
--local_ip_for_outbound_sockets=127.13.75.129
--webserver_interface=127.13.75.129
--webserver_port=0
--tserver_master_addrs=127.13.75.190:42209
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.129.keytab
--principal=kudu/127.13.75.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.75.148:43249
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175691491353-13614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175691491353-13614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175691491353-13614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:01:52.639626 17758 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:01:52.639840 17758 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:01:52.639868 17758 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:01:52.641180 17758 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:01:52.641228 17758 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:01:52.641258 17758 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.75.129
I20260923 15:01:52.642819 17758 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.75.148:43249
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175691491353-13614-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175691491353-13614-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.129.keytab
--principal=kudu/127.13.75.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.75.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175691491353-13614-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.75.129
--webserver_port=0
--tserver_master_addrs=127.13.75.190:42209
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17758
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.75.129
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175691491353-13614-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:01:52.643147 17758 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:01:52.643399 17758 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:01:52.649071 17764 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:01:52.649063 17766 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:01:52.649063 17763 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:01:52.649094 17758 server_base.cc:1034] running on GCE node
I20260923 15:01:52.649564 17758 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:01:52.649848 17758 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:01:52.650959 17758 hybrid_clock.cc:648] HybridClock initialized: now 1790175712650939 us; error 28 us; skew 500 ppm
Sep 23 15:01:52 dist-test-slave-cnb0 krb5kdc[17652](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175712, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:01:52.653870 17758 init.cc:377] Logged in from keytab as kudu/127.13.75.129@KRBTEST.COM (short username kudu)
I20260923 15:01:52.654253 17758 fs_manager.cc:359] Metadata directory not provided
I20260923 15:01:52.654336 17758 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:01:52.654505 17758 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:01:52.655731 17758 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175691491353-13614-0/minicluster-data/ts-0/data/instance:
uuid: "92215cf8e3704829aaec7d52d2d8f191"
format_stamp: "Formatted at 2026-09-23 15:01:52 on dist-test-slave-cnb0"
server_key: "13da44d2ccd888419e2bfed2c885453e"
server_key_iv: "5259436467912ce6958aa88a7de327ff"
server_key_version: "encryptionkey@0"
I20260923 15:01:52.656111 17758 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175691491353-13614-0/minicluster-data/ts-0/wal/instance:
uuid: "92215cf8e3704829aaec7d52d2d8f191"
format_stamp: "Formatted at 2026-09-23 15:01:52 on dist-test-slave-cnb0"
server_key: "13da44d2ccd888419e2bfed2c885453e"
server_key_iv: "5259436467912ce6958aa88a7de327ff"
server_key_version: "encryptionkey@0"
I20260923 15:01:52.657542 17758 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 15:01:52.658454 17772 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:01:52.658712 17758 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:01:52.658850 17758 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175691491353-13614-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175691491353-13614-0/minicluster-data/ts-0/wal
uuid: "92215cf8e3704829aaec7d52d2d8f191"
format_stamp: "Formatted at 2026-09-23 15:01:52 on dist-test-slave-cnb0"
server_key: "13da44d2ccd888419e2bfed2c885453e"
server_key_iv: "5259436467912ce6958aa88a7de327ff"
server_key_version: "encryptionkey@0"
I20260923 15:01:52.658998 17758 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175691491353-13614-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175691491353-13614-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175691491353-13614-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:01:52.678495 17758 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:01:52.681893 17758 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:01:52.682067 17758 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:01:52.682354 17758 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 15:01:52.682756 17758 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 15:01:52.682804 17758 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 15:01:52.682837 17758 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 15:01:52.682865 17758 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 15:01:52.689404 17758 rpc_server.cc:307] RPC server started. Bound to: 127.13.75.129:44285
I20260923 15:01:52.689505 17885 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.75.129:44285 every 8 connection(s)
I20260923 15:01:52.689774 17758 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175691491353-13614-0/minicluster-data/ts-0/data/info.pb
I20260923 15:01:52.691084 13614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKC4soy/build/release/bin/kudu as pid 17758
I20260923 15:01:52.691195 13614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175691491353-13614-0/minicluster-data/ts-0/wal/instance
I20260923 15:01:52.691391 13614 external_mini_cluster.cc:1428] Setting key 39f06ef8e6f2a26bb401d4f8e2af6f14
Sep 23 15:01:52 dist-test-slave-cnb0 krb5kdc[17652](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175712, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.129@KRBTEST.COM for kudu/127.13.75.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.13.75.130@KRBTEST.COM; defaulting to no policy
I20260923 15:01:52.701476 17736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:52.690552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.129:53635 (local address 127.13.75.190:42209)
0923 15:01:52.690639 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0923 15:01:52.690641 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:01:52.690828 (+ 187us) server_negotiation.cc:407] Connection header received
0923 15:01:52.691619 (+ 791us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:01:52.691621 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:01:52.691636 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:01:52.691704 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:01:52.693103 (+ 1399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:52.693696 (+ 593us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:52.694710 (+ 1014us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:52.695097 (+ 387us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:52.695178 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:01:52.697797 (+ 2619us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:01:52.697799 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:01:52.697800 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:01:52.697808 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:01:52.699666 (+ 1858us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:01:52.700144 (+ 478us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:01:52.700145 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:01:52.700145 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:01:52.700207 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:01:52.700659 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:01:52.700661 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:01:52.700661 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:01:52.700811 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:01:52.700873 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0923 15:01:52.701203 (+ 330us) server_negotiation.cc:299] Negotiation successful
0923 15:01:52.701288 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260923 15:01:52.701728 17888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:52.690523 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:42209 (local address 127.13.75.129:53635)
0923 15:01:52.690684 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0923 15:01:52.690727 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0923 15:01:52.691505 (+ 778us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:01:52.691870 (+ 365us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:01:52.691875 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:01:52.692260 (+ 385us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:01:52.692998 (+ 738us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:01:52.693006 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:01:52.693802 (+ 796us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:01:52.693804 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:01:52.694618 (+ 814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:01:52.694619 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:01:52.695212 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:01:52.695213 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:01:52.695324 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:01:52.695991 (+ 667us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:01:52.696002 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:01:52.697685 (+ 1683us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:01:52.699775 (+ 2090us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:01:52.699783 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:01:52.699785 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:01:52.700045 (+ 260us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:01:52.700322 (+ 277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:01:52.700323 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:01:52.700323 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:01:52.700572 (+ 249us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:01:52.700888 (+ 316us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:01:52.700897 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:01:52.701039 (+ 142us) client_negotiation.cc:769] Sending connection context
0923 15:01:52.701141 (+ 102us) client_negotiation.cc:240] Negotiation successful
0923 15:01:52.701301 (+ 160us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":98,"threads_started":1}
I20260923 15:01:52.702455 17886 heartbeater.cc:347] Connected to a master server at 127.13.75.190:42209
I20260923 15:01:52.702613 17886 heartbeater.cc:464] Registering TS with master...
I20260923 15:01:52.702826 17886 heartbeater.cc:511] Master 127.13.75.190:42209 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.75.130@KRBTEST.COM" created.
I20260923 15:01:52.703539 17696 ts_manager.cc:194] Registered new tserver with Master: 92215cf8e3704829aaec7d52d2d8f191 (127.13.75.129:44285)
I20260923 15:01:52.704252 17696 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.75.129@KRBTEST.COM'} at 127.13.75.129:53635
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.75.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.130.keytab.
Entry for principal kudu/127.13.75.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.130.keytab.
I20260923 15:01:52.723819 13614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKC4soy/build/release/bin/kudu
/tmp/dist-test-taskKC4soy/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175691491353-13614-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175691491353-13614-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175691491353-13614-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175691491353-13614-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.75.130:0
--local_ip_for_outbound_sockets=127.13.75.130
--webserver_interface=127.13.75.130
--webserver_port=0
--tserver_master_addrs=127.13.75.190:42209
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.130.keytab
--principal=kudu/127.13.75.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.75.148:43249
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175691491353-13614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175691491353-13614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175691491353-13614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:01:52.811669 17891 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:01:52.811946 17891 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:01:52.811976 17891 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:01:52.813755 17891 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:01:52.813814 17891 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:01:52.813848 17891 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.75.130
I20260923 15:01:52.815605 17891 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.75.148:43249
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175691491353-13614-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175691491353-13614-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.130.keytab
--principal=kudu/127.13.75.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.75.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175691491353-13614-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.75.130
--webserver_port=0
--tserver_master_addrs=127.13.75.190:42209
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.17891
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.75.130
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175691491353-13614-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:01:52.815889 17891 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:01:52.816165 17891 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:01:52.821791 17899 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:01:52.821807 17897 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:01:52.821808 17896 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:01:52.822176 17891 server_base.cc:1034] running on GCE node
I20260923 15:01:52.822413 17891 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:01:52.822683 17891 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:01:52.823850 17891 hybrid_clock.cc:648] HybridClock initialized: now 1790175712823822 us; error 62 us; skew 500 ppm
Sep 23 15:01:52 dist-test-slave-cnb0 krb5kdc[17652](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175712, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:01:52.827122 17891 init.cc:377] Logged in from keytab as kudu/127.13.75.130@KRBTEST.COM (short username kudu)
I20260923 15:01:52.827569 17891 fs_manager.cc:359] Metadata directory not provided
I20260923 15:01:52.827641 17891 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:01:52.827755 17891 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:01:52.828980 17891 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175691491353-13614-0/minicluster-data/ts-1/data/instance:
uuid: "8a2fbb1eaff540f1bc125b03da5a7fa1"
format_stamp: "Formatted at 2026-09-23 15:01:52 on dist-test-slave-cnb0"
server_key: "431f9c0db648a3b8756222ff8400d8eb"
server_key_iv: "379d8c1a23e766220674a3375bb536a4"
server_key_version: "encryptionkey@0"
I20260923 15:01:52.829347 17891 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175691491353-13614-0/minicluster-data/ts-1/wal/instance:
uuid: "8a2fbb1eaff540f1bc125b03da5a7fa1"
format_stamp: "Formatted at 2026-09-23 15:01:52 on dist-test-slave-cnb0"
server_key: "431f9c0db648a3b8756222ff8400d8eb"
server_key_iv: "379d8c1a23e766220674a3375bb536a4"
server_key_version: "encryptionkey@0"
I20260923 15:01:52.831089 17891 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260923 15:01:52.832075 17905 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:01:52.832386 17891 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:01:52.832471 17891 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175691491353-13614-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175691491353-13614-0/minicluster-data/ts-1/wal
uuid: "8a2fbb1eaff540f1bc125b03da5a7fa1"
format_stamp: "Formatted at 2026-09-23 15:01:52 on dist-test-slave-cnb0"
server_key: "431f9c0db648a3b8756222ff8400d8eb"
server_key_iv: "379d8c1a23e766220674a3375bb536a4"
server_key_version: "encryptionkey@0"
I20260923 15:01:52.832525 17891 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175691491353-13614-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175691491353-13614-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175691491353-13614-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:01:52.852237 17891 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:01:52.854633 17891 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:01:52.854750 17891 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:01:52.855023 17891 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 15:01:52.855497 17891 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 15:01:52.855590 17891 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 15:01:52.855619 17891 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 15:01:52.855693 17891 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 15:01:52.862740 17891 rpc_server.cc:307] RPC server started. Bound to: 127.13.75.130:45909
I20260923 15:01:52.862789 18018 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.75.130:45909 every 8 connection(s)
I20260923 15:01:52.863142 17891 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175691491353-13614-0/minicluster-data/ts-1/data/info.pb
I20260923 15:01:52.869702 13614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKC4soy/build/release/bin/kudu as pid 17891
I20260923 15:01:52.869848 13614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175691491353-13614-0/minicluster-data/ts-1/wal/instance
I20260923 15:01:52.870081 13614 external_mini_cluster.cc:1428] Setting key 6935b6279c6289925f4808d5ae2af2c1
Sep 23 15:01:52 dist-test-slave-cnb0 krb5kdc[17652](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175712, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.130@KRBTEST.COM for kudu/127.13.75.190@KRBTEST.COM
I20260923 15:01:52.874315 18021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:52.863922 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:42209 (local address 127.13.75.130:52923)
0923 15:01:52.864161 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0923 15:01:52.864212 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0923 15:01:52.864754 (+ 542us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:01:52.865041 (+ 287us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:01:52.865046 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:01:52.865341 (+ 295us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:01:52.865905 (+ 564us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:01:52.865911 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:01:52.866735 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:01:52.866737 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:01:52.867722 (+ 985us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:01:52.867725 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:01:52.868196 (+ 471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:01:52.868198 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:01:52.868335 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:01:52.868956 (+ 621us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:01:52.868966 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:01:52.870694 (+ 1728us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:01:52.872798 (+ 2104us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:01:52.872804 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:01:52.872806 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:01:52.873063 (+ 257us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:01:52.873340 (+ 277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:01:52.873341 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:01:52.873341 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:01:52.873464 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:01:52.873770 (+ 306us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:01:52.873775 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:01:52.873898 (+ 123us) client_negotiation.cc:769] Sending connection context
0923 15:01:52.874008 (+ 110us) client_negotiation.cc:240] Negotiation successful
0923 15:01:52.874140 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":200,"thread_start_us":115,"threads_started":1}
I20260923 15:01:52.874311 17736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:52.863989 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.130:52923 (local address 127.13.75.190:42209)
0923 15:01:52.864087 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0923 15:01:52.864090 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:01:52.864309 (+ 219us) server_negotiation.cc:407] Connection header received
0923 15:01:52.864887 (+ 578us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:01:52.864888 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:01:52.864903 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:01:52.864959 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:01:52.866042 (+ 1083us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:52.866614 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:52.867844 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:52.868081 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:52.868184 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:01:52.870816 (+ 2632us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:01:52.870817 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:01:52.870817 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:01:52.870826 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:01:52.872697 (+ 1871us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:01:52.873151 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:01:52.873152 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:01:52.873153 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:01:52.873226 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:01:52.873544 (+ 318us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:01:52.873546 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:01:52.873546 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:01:52.873698 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:01:52.873771 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0923 15:01:52.874061 (+ 290us) server_negotiation.cc:299] Negotiation successful
0923 15:01:52.874167 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":63}
I20260923 15:01:52.874943 18019 heartbeater.cc:347] Connected to a master server at 127.13.75.190:42209
I20260923 15:01:52.875065 18019 heartbeater.cc:464] Registering TS with master...
I20260923 15:01:52.875252 18019 heartbeater.cc:511] Master 127.13.75.190:42209 requested a full tablet report, sending...
I20260923 15:01:52.875658 17696 ts_manager.cc:194] Registered new tserver with Master: 8a2fbb1eaff540f1bc125b03da5a7fa1 (127.13.75.130:45909)
I20260923 15:01:52.876070 17696 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.75.130@KRBTEST.COM'} at 127.13.75.130:52923
WARNING: no policy specified for kudu/127.13.75.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.75.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.75.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.131.keytab.
Entry for principal kudu/127.13.75.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.131.keytab.
I20260923 15:01:52.903281 13614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKC4soy/build/release/bin/kudu
/tmp/dist-test-taskKC4soy/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175691491353-13614-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175691491353-13614-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175691491353-13614-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175691491353-13614-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.75.131:0
--local_ip_for_outbound_sockets=127.13.75.131
--webserver_interface=127.13.75.131
--webserver_port=0
--tserver_master_addrs=127.13.75.190:42209
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.131.keytab
--principal=kudu/127.13.75.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.75.148:43249
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--webserver_enabled=0 with env {KRB5CCNAME=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175691491353-13614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175691491353-13614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175691491353-13614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:01:52.986764 18024 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:01:52.987062 18024 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:01:52.987094 18024 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:01:52.988327 18024 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:01:52.988370 18024 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:01:52.988399 18024 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.75.131
I20260923 15:01:52.989924 18024 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.75.148:43249
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175691491353-13614-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175691491353-13614-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.131.keytab
--principal=kudu/127.13.75.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.75.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--webserver_enabled=false
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175691491353-13614-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.75.131
--webserver_port=0
--tserver_master_addrs=127.13.75.190:42209
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18024
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.75.131
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175691491353-13614-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:01:52.990190 18024 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:01:52.990455 18024 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:01:52.995831 18030 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:01:52.995832 18032 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:01:52.995836 18029 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:01:52.996133 18024 server_base.cc:1034] running on GCE node
I20260923 15:01:52.996384 18024 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:01:52.996658 18024 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:01:52.997885 18024 hybrid_clock.cc:648] HybridClock initialized: now 1790175712997830 us; error 67 us; skew 500 ppm
Sep 23 15:01:52 dist-test-slave-cnb0 krb5kdc[17652](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175712, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:01:53.000880 18024 init.cc:377] Logged in from keytab as kudu/127.13.75.131@KRBTEST.COM (short username kudu)
I20260923 15:01:53.001291 18024 fs_manager.cc:359] Metadata directory not provided
I20260923 15:01:53.001344 18024 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:01:53.001529 18024 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:01:53.002677 18024 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175691491353-13614-0/minicluster-data/ts-2/data/instance:
uuid: "954ac86dadd5427a91ced035a4d099b4"
format_stamp: "Formatted at 2026-09-23 15:01:52 on dist-test-slave-cnb0"
server_key: "d4f6863c8533b208d99dc4805bcbfd38"
server_key_iv: "7387e0d7a128d4510a245fc4fb0936c1"
server_key_version: "encryptionkey@0"
I20260923 15:01:53.003098 18024 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175691491353-13614-0/minicluster-data/ts-2/wal/instance:
uuid: "954ac86dadd5427a91ced035a4d099b4"
format_stamp: "Formatted at 2026-09-23 15:01:52 on dist-test-slave-cnb0"
server_key: "d4f6863c8533b208d99dc4805bcbfd38"
server_key_iv: "7387e0d7a128d4510a245fc4fb0936c1"
server_key_version: "encryptionkey@0"
I20260923 15:01:53.004645 18024 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260923 15:01:53.005714 18038 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:01:53.005967 18024 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260923 15:01:53.006091 18024 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175691491353-13614-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175691491353-13614-0/minicluster-data/ts-2/wal
uuid: "954ac86dadd5427a91ced035a4d099b4"
format_stamp: "Formatted at 2026-09-23 15:01:52 on dist-test-slave-cnb0"
server_key: "d4f6863c8533b208d99dc4805bcbfd38"
server_key_iv: "7387e0d7a128d4510a245fc4fb0936c1"
server_key_version: "encryptionkey@0"
I20260923 15:01:53.006194 18024 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175691491353-13614-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175691491353-13614-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175691491353-13614-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:01:53.030092 18024 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:01:53.032917 18024 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:01:53.033155 18024 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:01:53.033452 18024 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 15:01:53.033798 18024 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 15:01:53.033859 18024 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 15:01:53.033902 18024 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 15:01:53.033923 18024 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 15:01:53.040623 18024 rpc_server.cc:307] RPC server started. Bound to: 127.13.75.131:43899
I20260923 15:01:53.040654 18151 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.75.131:43899 every 8 connection(s)
I20260923 15:01:53.040951 18024 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175691491353-13614-0/minicluster-data/ts-2/data/info.pb
Sep 23 15:01:53 dist-test-slave-cnb0 krb5kdc[17652](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175712, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.131@KRBTEST.COM for kudu/127.13.75.190@KRBTEST.COM
I20260923 15:01:53.048905 13614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKC4soy/build/release/bin/kudu as pid 18024
I20260923 15:01:53.049011 13614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175691491353-13614-0/minicluster-data/ts-2/wal/instance
I20260923 15:01:53.049211 13614 external_mini_cluster.cc:1428] Setting key fedcac16af199822f3b7eeaa71e1d712
I20260923 15:01:53.052572 17736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:53.041857 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.131:55263 (local address 127.13.75.190:42209)
0923 15:01:53.041926 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0923 15:01:53.041928 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:01:53.042056 (+ 128us) server_negotiation.cc:407] Connection header received
0923 15:01:53.042619 (+ 563us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:01:53.042620 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:01:53.042633 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:01:53.042705 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:01:53.043910 (+ 1205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:53.044458 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:53.045687 (+ 1229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:53.045873 (+ 186us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:53.045939 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:01:53.048419 (+ 2480us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:01:53.048420 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:01:53.048420 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:01:53.048429 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:01:53.050270 (+ 1841us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:01:53.050776 (+ 506us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:01:53.050777 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:01:53.050778 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:01:53.050836 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:01:53.051657 (+ 821us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:01:53.051658 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:01:53.051659 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:01:53.051825 (+ 166us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:01:53.051895 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0923 15:01:53.052354 (+ 459us) server_negotiation.cc:299] Negotiation successful
0923 15:01:53.052497 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260923 15:01:53.052829 18154 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:53.041703 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:42209 (local address 127.13.75.131:55263)
0923 15:01:53.041911 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0923 15:01:53.041959 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0923 15:01:53.042464 (+ 505us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:01:53.042792 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:01:53.042796 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:01:53.043173 (+ 377us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:01:53.043781 (+ 608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:01:53.043788 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:01:53.044576 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:01:53.044577 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:01:53.045588 (+ 1011us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:01:53.045591 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:01:53.045965 (+ 374us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:01:53.045966 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:01:53.046089 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:01:53.046690 (+ 601us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:01:53.046699 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:01:53.048278 (+ 1579us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:01:53.050385 (+ 2107us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:01:53.050392 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:01:53.050395 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:01:53.050693 (+ 298us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:01:53.050995 (+ 302us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:01:53.050996 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:01:53.050997 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:01:53.051432 (+ 435us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:01:53.051948 (+ 516us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:01:53.051954 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:01:53.052131 (+ 177us) client_negotiation.cc:769] Sending connection context
0923 15:01:53.052234 (+ 103us) client_negotiation.cc:240] Negotiation successful
0923 15:01:53.052376 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":111,"threads_started":1}
I20260923 15:01:53.053483 18152 heartbeater.cc:347] Connected to a master server at 127.13.75.190:42209
I20260923 15:01:53.053588 18152 heartbeater.cc:464] Registering TS with master...
I20260923 15:01:53.053716 18152 heartbeater.cc:511] Master 127.13.75.190:42209 requested a full tablet report, sending...
I20260923 15:01:53.054147 17696 ts_manager.cc:194] Registered new tserver with Master: 954ac86dadd5427a91ced035a4d099b4 (127.13.75.131:43899)
I20260923 15:01:53.054535 17696 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.75.131@KRBTEST.COM'} at 127.13.75.131:55263
I20260923 15:01:53.061550 13614 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260923 15:01:53.069651 17736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:53.062997 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44302 (local address 127.13.75.190:42209)
0923 15:01:53.063053 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0923 15:01:53.063056 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:01:53.063175 (+ 119us) server_negotiation.cc:407] Connection header received
0923 15:01:53.063247 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:01:53.063248 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:01:53.063257 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:01:53.063305 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:01:53.063959 (+ 654us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:53.064502 (+ 543us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:53.065222 (+ 720us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:53.065433 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:53.065500 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:01:53.066810 (+ 1310us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:01:53.066811 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:01:53.066812 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:01:53.066819 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:01:53.068309 (+ 1490us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:01:53.068692 (+ 383us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:01:53.068693 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:01:53.068693 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:01:53.068747 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:01:53.069020 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:01:53.069021 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:01:53.069021 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:01:53.069191 (+ 170us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:01:53.069277 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0923 15:01:53.069419 (+ 142us) server_negotiation.cc:299] Negotiation successful
0923 15:01:53.069487 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260923 15:01:53.071686 17696 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44302:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 15:01:53.073279 17696 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 15:01:53.080948 18166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:53.076403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.131:43899 (local address 127.0.0.1:55544)
0923 15:01:53.076905 (+ 502us) negotiation.cc:107] Waiting for socket to connect
0923 15:01:53.076927 (+ 22us) client_negotiation.cc:174] Beginning negotiation
0923 15:01:53.076981 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:01:53.077286 (+ 305us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:01:53.077287 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:01:53.077290 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:01:53.077335 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:01:53.077337 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:01:53.078583 (+ 1246us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:01:53.078584 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:01:53.079568 (+ 984us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:01:53.079571 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:01:53.080590 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:01:53.080591 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:01:53.080682 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:01:53.080701 (+ 19us) client_negotiation.cc:769] Sending connection context
0923 15:01:53.080786 (+ 85us) client_negotiation.cc:240] Negotiation successful
0923 15:01:53.080853 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":482,"mutex_wait_us":22,"thread_start_us":78,"threads_started":1}
I20260923 15:01:53.080953 18168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:53.076622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55544 (local address 127.13.75.131:43899)
0923 15:01:53.076818 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0923 15:01:53.076855 (+ 37us) server_negotiation.cc:399] Waiting for connection header
0923 15:01:53.077044 (+ 189us) server_negotiation.cc:407] Connection header received
0923 15:01:53.077091 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:01:53.077094 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:01:53.077113 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:01:53.077188 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:01:53.077447 (+ 259us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:53.078477 (+ 1030us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:53.079669 (+ 1192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:53.080472 (+ 803us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:53.080556 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:01:53.080620 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0923 15:01:53.080813 (+ 193us) server_negotiation.cc:299] Negotiation successful
0923 15:01:53.080858 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":167,"thread_start_us":90,"threads_started":1}
I20260923 15:01:53.081192 18165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:53.076322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40534 (local address 127.13.75.129:44285)
0923 15:01:53.076517 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0923 15:01:53.076523 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 15:01:53.076729 (+ 206us) server_negotiation.cc:407] Connection header received
0923 15:01:53.076773 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:01:53.076776 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:01:53.076798 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:01:53.076876 (+ 78us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:01:53.077209 (+ 333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:53.078058 (+ 849us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:53.079790 (+ 1732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:53.080477 (+ 687us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:53.080563 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:01:53.080620 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0923 15:01:53.081048 (+ 428us) server_negotiation.cc:299] Negotiation successful
0923 15:01:53.081100 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":86,"threads_started":1}
I20260923 15:01:53.081431 18164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:53.076189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.129:44285 (local address 127.0.0.1:40534)
0923 15:01:53.076629 (+ 440us) negotiation.cc:107] Waiting for socket to connect
0923 15:01:53.076640 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 15:01:53.076699 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:01:53.076992 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:01:53.076995 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:01:53.076998 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:01:53.077103 (+ 105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:01:53.077105 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:01:53.078176 (+ 1071us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:01:53.078178 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:01:53.079692 (+ 1514us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:01:53.079693 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:01:53.080717 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:01:53.080718 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:01:53.080933 (+ 215us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:01:53.080938 (+ 5us) client_negotiation.cc:769] Sending connection context
0923 15:01:53.081311 (+ 373us) client_negotiation.cc:240] Negotiation successful
0923 15:01:53.081361 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":421,"mutex_wait_us":41,"thread_start_us":86,"threads_started":1}
I20260923 15:01:53.081904 18163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:53.076189 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.130:45909 (local address 127.0.0.1:45792)
0923 15:01:53.076373 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0923 15:01:53.076434 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0923 15:01:53.076507 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:01:53.076992 (+ 485us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:01:53.076995 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:01:53.077000 (+ 5us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:01:53.077098 (+ 98us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:01:53.077100 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:01:53.078425 (+ 1325us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:01:53.078427 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:01:53.079690 (+ 1263us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:01:53.079693 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:01:53.081652 (+ 1959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:01:53.081653 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:01:53.081717 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:01:53.081721 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 15:01:53.081774 (+ 53us) client_negotiation.cc:240] Negotiation successful
0923 15:01:53.081821 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"mutex_wait_us":6,"thread_start_us":65,"threads_started":1}
I20260923 15:01:53.082252 18167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:53.076412 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45792 (local address 127.13.75.130:45909)
0923 15:01:53.076608 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0923 15:01:53.076612 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 15:01:53.076623 (+ 11us) server_negotiation.cc:407] Connection header received
0923 15:01:53.076664 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:01:53.076666 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:01:53.076682 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:01:53.076880 (+ 198us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:01:53.077189 (+ 309us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:53.078294 (+ 1105us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:53.080664 (+ 2370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:53.081427 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:53.081502 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:01:53.081565 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0923 15:01:53.082133 (+ 568us) server_negotiation.cc:299] Negotiation successful
0923 15:01:53.082183 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":167,"thread_start_us":139,"threads_started":1}
I20260923 15:01:53.082195 18086 tablet_service.cc:1467] Processing CreateTablet for tablet 8656560a80ca4b789e1e2f0e23efc224 (DEFAULT_TABLE table=test-table [id=c8e5af0becd64a82b552363fd5c27762]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 15:01:53.082582 18086 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8656560a80ca4b789e1e2f0e23efc224. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:01:53.083020 17953 tablet_service.cc:1467] Processing CreateTablet for tablet 8656560a80ca4b789e1e2f0e23efc224 (DEFAULT_TABLE table=test-table [id=c8e5af0becd64a82b552363fd5c27762]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 15:01:53.083341 17953 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8656560a80ca4b789e1e2f0e23efc224. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:01:53.085263 17820 tablet_service.cc:1467] Processing CreateTablet for tablet 8656560a80ca4b789e1e2f0e23efc224 (DEFAULT_TABLE table=test-table [id=c8e5af0becd64a82b552363fd5c27762]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 15:01:53.085592 17820 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8656560a80ca4b789e1e2f0e23efc224. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:01:53.086558 18169 tablet_bootstrap.cc:492] T 8656560a80ca4b789e1e2f0e23efc224 P 8a2fbb1eaff540f1bc125b03da5a7fa1: Bootstrap starting.
I20260923 15:01:53.087299 18169 tablet_bootstrap.cc:654] T 8656560a80ca4b789e1e2f0e23efc224 P 8a2fbb1eaff540f1bc125b03da5a7fa1: Neither blocks nor log segments found. Creating new log.
I20260923 15:01:53.087951 18169 log.cc:824] T 8656560a80ca4b789e1e2f0e23efc224 P 8a2fbb1eaff540f1bc125b03da5a7fa1: Log is configured to *not* fsync() on all Append() calls
I20260923 15:01:53.088332 18170 tablet_bootstrap.cc:492] T 8656560a80ca4b789e1e2f0e23efc224 P 954ac86dadd5427a91ced035a4d099b4: Bootstrap starting.
I20260923 15:01:53.089120 18170 tablet_bootstrap.cc:654] T 8656560a80ca4b789e1e2f0e23efc224 P 954ac86dadd5427a91ced035a4d099b4: Neither blocks nor log segments found. Creating new log.
I20260923 15:01:53.089275 18169 tablet_bootstrap.cc:492] T 8656560a80ca4b789e1e2f0e23efc224 P 8a2fbb1eaff540f1bc125b03da5a7fa1: No bootstrap required, opened a new log
I20260923 15:01:53.089308 18172 tablet_bootstrap.cc:492] T 8656560a80ca4b789e1e2f0e23efc224 P 92215cf8e3704829aaec7d52d2d8f191: Bootstrap starting.
I20260923 15:01:53.089356 18169 ts_tablet_manager.cc:1397] T 8656560a80ca4b789e1e2f0e23efc224 P 8a2fbb1eaff540f1bc125b03da5a7fa1: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260923 15:01:53.089826 18170 log.cc:824] T 8656560a80ca4b789e1e2f0e23efc224 P 954ac86dadd5427a91ced035a4d099b4: Log is configured to *not* fsync() on all Append() calls
I20260923 15:01:53.090075 18172 tablet_bootstrap.cc:654] T 8656560a80ca4b789e1e2f0e23efc224 P 92215cf8e3704829aaec7d52d2d8f191: Neither blocks nor log segments found. Creating new log.
I20260923 15:01:53.090739 18172 log.cc:824] T 8656560a80ca4b789e1e2f0e23efc224 P 92215cf8e3704829aaec7d52d2d8f191: Log is configured to *not* fsync() on all Append() calls
I20260923 15:01:53.090950 18170 tablet_bootstrap.cc:492] T 8656560a80ca4b789e1e2f0e23efc224 P 954ac86dadd5427a91ced035a4d099b4: No bootstrap required, opened a new log
I20260923 15:01:53.091125 18170 ts_tablet_manager.cc:1397] T 8656560a80ca4b789e1e2f0e23efc224 P 954ac86dadd5427a91ced035a4d099b4: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.001s
I20260923 15:01:53.091459 18169 raft_consensus.cc:348] T 8656560a80ca4b789e1e2f0e23efc224 P 8a2fbb1eaff540f1bc125b03da5a7fa1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92215cf8e3704829aaec7d52d2d8f191" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 44285 } } peers { permanent_uuid: "8a2fbb1eaff540f1bc125b03da5a7fa1" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 45909 } } peers { permanent_uuid: "954ac86dadd5427a91ced035a4d099b4" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 43899 } }
I20260923 15:01:53.091629 18169 raft_consensus.cc:374] T 8656560a80ca4b789e1e2f0e23efc224 P 8a2fbb1eaff540f1bc125b03da5a7fa1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:01:53.091673 18169 raft_consensus.cc:729] T 8656560a80ca4b789e1e2f0e23efc224 P 8a2fbb1eaff540f1bc125b03da5a7fa1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8a2fbb1eaff540f1bc125b03da5a7fa1, State: Initialized, Role: FOLLOWER
I20260923 15:01:53.091756 18172 tablet_bootstrap.cc:492] T 8656560a80ca4b789e1e2f0e23efc224 P 92215cf8e3704829aaec7d52d2d8f191: No bootstrap required, opened a new log
I20260923 15:01:53.091787 18169 consensus_queue.cc:260] T 8656560a80ca4b789e1e2f0e23efc224 P 8a2fbb1eaff540f1bc125b03da5a7fa1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92215cf8e3704829aaec7d52d2d8f191" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 44285 } } peers { permanent_uuid: "8a2fbb1eaff540f1bc125b03da5a7fa1" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 45909 } } peers { permanent_uuid: "954ac86dadd5427a91ced035a4d099b4" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 43899 } }
I20260923 15:01:53.091840 18172 ts_tablet_manager.cc:1397] T 8656560a80ca4b789e1e2f0e23efc224 P 92215cf8e3704829aaec7d52d2d8f191: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260923 15:01:53.092082 18019 heartbeater.cc:503] Master 127.13.75.190:42209 was elected leader, sending a full tablet report...
I20260923 15:01:53.092100 18169 ts_tablet_manager.cc:1428] T 8656560a80ca4b789e1e2f0e23efc224 P 8a2fbb1eaff540f1bc125b03da5a7fa1: Time spent starting tablet: real 0.003s user 0.002s sys 0.001s
I20260923 15:01:53.093603 18170 raft_consensus.cc:348] T 8656560a80ca4b789e1e2f0e23efc224 P 954ac86dadd5427a91ced035a4d099b4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92215cf8e3704829aaec7d52d2d8f191" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 44285 } } peers { permanent_uuid: "8a2fbb1eaff540f1bc125b03da5a7fa1" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 45909 } } peers { permanent_uuid: "954ac86dadd5427a91ced035a4d099b4" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 43899 } }
I20260923 15:01:53.093792 18170 raft_consensus.cc:374] T 8656560a80ca4b789e1e2f0e23efc224 P 954ac86dadd5427a91ced035a4d099b4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:01:53.093832 18170 raft_consensus.cc:729] T 8656560a80ca4b789e1e2f0e23efc224 P 954ac86dadd5427a91ced035a4d099b4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 954ac86dadd5427a91ced035a4d099b4, State: Initialized, Role: FOLLOWER
I20260923 15:01:53.093971 18170 consensus_queue.cc:260] T 8656560a80ca4b789e1e2f0e23efc224 P 954ac86dadd5427a91ced035a4d099b4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92215cf8e3704829aaec7d52d2d8f191" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 44285 } } peers { permanent_uuid: "8a2fbb1eaff540f1bc125b03da5a7fa1" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 45909 } } peers { permanent_uuid: "954ac86dadd5427a91ced035a4d099b4" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 43899 } }
I20260923 15:01:53.094300 18170 ts_tablet_manager.cc:1428] T 8656560a80ca4b789e1e2f0e23efc224 P 954ac86dadd5427a91ced035a4d099b4: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260923 15:01:53.094321 18152 heartbeater.cc:503] Master 127.13.75.190:42209 was elected leader, sending a full tablet report...
I20260923 15:01:53.094213 18172 raft_consensus.cc:348] T 8656560a80ca4b789e1e2f0e23efc224 P 92215cf8e3704829aaec7d52d2d8f191 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92215cf8e3704829aaec7d52d2d8f191" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 44285 } } peers { permanent_uuid: "8a2fbb1eaff540f1bc125b03da5a7fa1" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 45909 } } peers { permanent_uuid: "954ac86dadd5427a91ced035a4d099b4" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 43899 } }
I20260923 15:01:53.094395 18172 raft_consensus.cc:374] T 8656560a80ca4b789e1e2f0e23efc224 P 92215cf8e3704829aaec7d52d2d8f191 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:01:53.094444 18172 raft_consensus.cc:729] T 8656560a80ca4b789e1e2f0e23efc224 P 92215cf8e3704829aaec7d52d2d8f191 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 92215cf8e3704829aaec7d52d2d8f191, State: Initialized, Role: FOLLOWER
I20260923 15:01:53.094570 18172 consensus_queue.cc:260] T 8656560a80ca4b789e1e2f0e23efc224 P 92215cf8e3704829aaec7d52d2d8f191 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92215cf8e3704829aaec7d52d2d8f191" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 44285 } } peers { permanent_uuid: "8a2fbb1eaff540f1bc125b03da5a7fa1" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 45909 } } peers { permanent_uuid: "954ac86dadd5427a91ced035a4d099b4" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 43899 } }
I20260923 15:01:53.094935 18172 ts_tablet_manager.cc:1428] T 8656560a80ca4b789e1e2f0e23efc224 P 92215cf8e3704829aaec7d52d2d8f191: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260923 15:01:53.094978 17886 heartbeater.cc:503] Master 127.13.75.190:42209 was elected leader, sending a full tablet report...
W20260923 15:01:53.113760 18020 tablet.cc:2369] T 8656560a80ca4b789e1e2f0e23efc224 P 8a2fbb1eaff540f1bc125b03da5a7fa1: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 15:01:53.114727 18175 raft_consensus.cc:482] T 8656560a80ca4b789e1e2f0e23efc224 P 8a2fbb1eaff540f1bc125b03da5a7fa1 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 15:01:53.114809 18175 raft_consensus.cc:504] T 8656560a80ca4b789e1e2f0e23efc224 P 8a2fbb1eaff540f1bc125b03da5a7fa1 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92215cf8e3704829aaec7d52d2d8f191" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 44285 } } peers { permanent_uuid: "8a2fbb1eaff540f1bc125b03da5a7fa1" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 45909 } } peers { permanent_uuid: "954ac86dadd5427a91ced035a4d099b4" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 43899 } }
I20260923 15:01:53.115244 18175 leader_election.cc:288] T 8656560a80ca4b789e1e2f0e23efc224 P 8a2fbb1eaff540f1bc125b03da5a7fa1 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 92215cf8e3704829aaec7d52d2d8f191 (127.13.75.129:44285), 954ac86dadd5427a91ced035a4d099b4 (127.13.75.131:43899)
I20260923 15:01:53.118839 18021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:53.115570 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.129:44285 (local address 127.13.75.130:35515)
0923 15:01:53.115673 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0923 15:01:53.115688 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0923 15:01:53.115749 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:01:53.115923 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:01:53.115925 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:01:53.115947 (+ 22us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:01:53.116029 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:01:53.116031 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:01:53.116844 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:01:53.116846 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:01:53.117765 (+ 919us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:01:53.117767 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:01:53.118534 (+ 767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:01:53.118535 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:01:53.118621 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:01:53.118625 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 15:01:53.118705 (+ 80us) client_negotiation.cc:240] Negotiation successful
0923 15:01:53.118744 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":74}
I20260923 15:01:53.119050 18165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:53.115745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.130:35515 (local address 127.13.75.129:44285)
0923 15:01:53.115796 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0923 15:01:53.115799 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:01:53.115806 (+ 7us) server_negotiation.cc:407] Connection header received
0923 15:01:53.115837 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:01:53.115838 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:01:53.115850 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:01:53.115917 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:01:53.116150 (+ 233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:53.116729 (+ 579us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:53.117893 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:53.118432 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:53.118495 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:01:53.118504 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0923 15:01:53.118749 (+ 245us) server_negotiation.cc:299] Negotiation successful
0923 15:01:53.118967 (+ 218us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260923 15:01:53.119243 18178 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:53.115699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.131:43899 (local address 127.13.75.130:41973)
0923 15:01:53.115882 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0923 15:01:53.115893 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 15:01:53.115945 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:01:53.116150 (+ 205us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:01:53.116151 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:01:53.116153 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:01:53.116222 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:01:53.116224 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:01:53.117107 (+ 883us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:01:53.117108 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:01:53.118099 (+ 991us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:01:53.118102 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:01:53.118947 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:01:53.118948 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:01:53.119026 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:01:53.119031 (+ 5us) client_negotiation.cc:769] Sending connection context
0923 15:01:53.119103 (+ 72us) client_negotiation.cc:240] Negotiation successful
0923 15:01:53.119152 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166,"mutex_wait_us":3,"thread_start_us":110,"threads_started":1}
I20260923 15:01:53.119247 18168 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:53.115772 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.130:41973 (local address 127.13.75.131:43899)
0923 15:01:53.115823 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0923 15:01:53.115825 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:01:53.115963 (+ 138us) server_negotiation.cc:407] Connection header received
0923 15:01:53.115994 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:01:53.115995 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:01:53.116007 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:01:53.116074 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:01:53.116350 (+ 276us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:53.116968 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:53.118229 (+ 1261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:53.118737 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:53.118801 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:01:53.118809 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0923 15:01:53.119127 (+ 318us) server_negotiation.cc:299] Negotiation successful
0923 15:01:53.119182 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260923 15:01:53.119360 17840 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8656560a80ca4b789e1e2f0e23efc224" candidate_uuid: "8a2fbb1eaff540f1bc125b03da5a7fa1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "92215cf8e3704829aaec7d52d2d8f191" is_pre_election: true
I20260923 15:01:53.119537 17840 raft_consensus.cc:2454] T 8656560a80ca4b789e1e2f0e23efc224 P 92215cf8e3704829aaec7d52d2d8f191 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a2fbb1eaff540f1bc125b03da5a7fa1 in term 0.
I20260923 15:01:53.119702 18106 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8656560a80ca4b789e1e2f0e23efc224" candidate_uuid: "8a2fbb1eaff540f1bc125b03da5a7fa1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "954ac86dadd5427a91ced035a4d099b4" is_pre_election: true
I20260923 15:01:53.119900 18106 raft_consensus.cc:2454] T 8656560a80ca4b789e1e2f0e23efc224 P 954ac86dadd5427a91ced035a4d099b4 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 8a2fbb1eaff540f1bc125b03da5a7fa1 in term 0.
I20260923 15:01:53.119920 17909 leader_election.cc:302] T 8656560a80ca4b789e1e2f0e23efc224 P 8a2fbb1eaff540f1bc125b03da5a7fa1 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a2fbb1eaff540f1bc125b03da5a7fa1, 92215cf8e3704829aaec7d52d2d8f191; no voters:
I20260923 15:01:53.120127 18175 raft_consensus.cc:2781] T 8656560a80ca4b789e1e2f0e23efc224 P 8a2fbb1eaff540f1bc125b03da5a7fa1 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 15:01:53.120191 18175 raft_consensus.cc:482] T 8656560a80ca4b789e1e2f0e23efc224 P 8a2fbb1eaff540f1bc125b03da5a7fa1 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 15:01:53.120208 18175 raft_consensus.cc:3037] T 8656560a80ca4b789e1e2f0e23efc224 P 8a2fbb1eaff540f1bc125b03da5a7fa1 [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:01:53.120923 18175 raft_consensus.cc:504] T 8656560a80ca4b789e1e2f0e23efc224 P 8a2fbb1eaff540f1bc125b03da5a7fa1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92215cf8e3704829aaec7d52d2d8f191" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 44285 } } peers { permanent_uuid: "8a2fbb1eaff540f1bc125b03da5a7fa1" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 45909 } } peers { permanent_uuid: "954ac86dadd5427a91ced035a4d099b4" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 43899 } }
I20260923 15:01:53.121032 18175 leader_election.cc:288] T 8656560a80ca4b789e1e2f0e23efc224 P 8a2fbb1eaff540f1bc125b03da5a7fa1 [CANDIDATE]: Term 1 election: Requested vote from peers 92215cf8e3704829aaec7d52d2d8f191 (127.13.75.129:44285), 954ac86dadd5427a91ced035a4d099b4 (127.13.75.131:43899)
I20260923 15:01:53.121266 17840 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8656560a80ca4b789e1e2f0e23efc224" candidate_uuid: "8a2fbb1eaff540f1bc125b03da5a7fa1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "92215cf8e3704829aaec7d52d2d8f191"
I20260923 15:01:53.121335 18106 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8656560a80ca4b789e1e2f0e23efc224" candidate_uuid: "8a2fbb1eaff540f1bc125b03da5a7fa1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "954ac86dadd5427a91ced035a4d099b4"
I20260923 15:01:53.121368 17840 raft_consensus.cc:3037] T 8656560a80ca4b789e1e2f0e23efc224 P 92215cf8e3704829aaec7d52d2d8f191 [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:01:53.121405 18106 raft_consensus.cc:3037] T 8656560a80ca4b789e1e2f0e23efc224 P 954ac86dadd5427a91ced035a4d099b4 [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:01:53.122282 17840 raft_consensus.cc:2454] T 8656560a80ca4b789e1e2f0e23efc224 P 92215cf8e3704829aaec7d52d2d8f191 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a2fbb1eaff540f1bc125b03da5a7fa1 in term 1.
I20260923 15:01:53.122282 18106 raft_consensus.cc:2454] T 8656560a80ca4b789e1e2f0e23efc224 P 954ac86dadd5427a91ced035a4d099b4 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 8a2fbb1eaff540f1bc125b03da5a7fa1 in term 1.
I20260923 15:01:53.122480 17909 leader_election.cc:302] T 8656560a80ca4b789e1e2f0e23efc224 P 8a2fbb1eaff540f1bc125b03da5a7fa1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8a2fbb1eaff540f1bc125b03da5a7fa1, 954ac86dadd5427a91ced035a4d099b4; no voters:
I20260923 15:01:53.122570 18175 raft_consensus.cc:2781] T 8656560a80ca4b789e1e2f0e23efc224 P 8a2fbb1eaff540f1bc125b03da5a7fa1 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 15:01:53.122725 18175 raft_consensus.cc:686] T 8656560a80ca4b789e1e2f0e23efc224 P 8a2fbb1eaff540f1bc125b03da5a7fa1 [term 1 LEADER]: Becoming Leader. State: Replica: 8a2fbb1eaff540f1bc125b03da5a7fa1, State: Running, Role: LEADER
I20260923 15:01:53.122808 18175 consensus_queue.cc:237] T 8656560a80ca4b789e1e2f0e23efc224 P 8a2fbb1eaff540f1bc125b03da5a7fa1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92215cf8e3704829aaec7d52d2d8f191" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 44285 } } peers { permanent_uuid: "8a2fbb1eaff540f1bc125b03da5a7fa1" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 45909 } } peers { permanent_uuid: "954ac86dadd5427a91ced035a4d099b4" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 43899 } }
I20260923 15:01:53.123755 17695 catalog_manager.cc:5495] T 8656560a80ca4b789e1e2f0e23efc224 P 8a2fbb1eaff540f1bc125b03da5a7fa1 reported cstate change: term changed from 0 to 1, leader changed from <none> to 8a2fbb1eaff540f1bc125b03da5a7fa1 (127.13.75.130). New cstate: current_term: 1 leader_uuid: "8a2fbb1eaff540f1bc125b03da5a7fa1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "92215cf8e3704829aaec7d52d2d8f191" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 44285 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8a2fbb1eaff540f1bc125b03da5a7fa1" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 45909 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "954ac86dadd5427a91ced035a4d099b4" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 43899 } health_report { overall_health: UNKNOWN } } }
I20260923 15:01:53.141289 18167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:53.138441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45794 (local address 127.13.75.130:45909)
0923 15:01:53.138521 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0923 15:01:53.138523 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:01:53.138536 (+ 13us) server_negotiation.cc:407] Connection header received
0923 15:01:53.138609 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:01:53.138610 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:01:53.138628 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:01:53.138710 (+ 82us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 15:01:53.138927 (+ 217us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:53.139455 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:53.140372 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:53.140591 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:53.140700 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:01:53.140893 (+ 193us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 15:01:53.141016 (+ 123us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 15:01:53.141136 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0923 15:01:53.141163 (+ 27us) server_negotiation.cc:299] Negotiation successful
0923 15:01:53.141186 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":52}
I20260923 15:01:53.143340 18106 raft_consensus.cc:1261] T 8656560a80ca4b789e1e2f0e23efc224 P 954ac86dadd5427a91ced035a4d099b4 [term 1 FOLLOWER]: Refusing update from remote peer 8a2fbb1eaff540f1bc125b03da5a7fa1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 15:01:53.143385 17840 raft_consensus.cc:1261] T 8656560a80ca4b789e1e2f0e23efc224 P 92215cf8e3704829aaec7d52d2d8f191 [term 1 FOLLOWER]: Refusing update from remote peer 8a2fbb1eaff540f1bc125b03da5a7fa1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 15:01:53.143774 18175 consensus_queue.cc:1035] T 8656560a80ca4b789e1e2f0e23efc224 P 8a2fbb1eaff540f1bc125b03da5a7fa1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "954ac86dadd5427a91ced035a4d099b4" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 43899 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 15:01:53.143954 18179 consensus_queue.cc:1035] T 8656560a80ca4b789e1e2f0e23efc224 P 8a2fbb1eaff540f1bc125b03da5a7fa1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "92215cf8e3704829aaec7d52d2d8f191" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 44285 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 15:01:53.146929 18184 mvcc.cc:205] Tried to move back new op lower bound from 7332559721031905280 to 7332559720951730176. Current Snapshot: MvccSnapshot[applied={T|T < 7332559721031905280}]
I20260923 15:01:53.147459 18183 mvcc.cc:205] Tried to move back new op lower bound from 7332559721031905280 to 7332559720951730176. Current Snapshot: MvccSnapshot[applied={T|T < 7332559721031905280}]
I20260923 15:01:53.149123 17695 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44302:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 15:01:53.149225 17695 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44302:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 15:01:53.150045 17695 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 4bb808cd65f94e19a9433ece9eb1281a: Sending DeleteTablet for 3 replicas of tablet 8656560a80ca4b789e1e2f0e23efc224
I20260923 15:01:53.150460 17820 tablet_service.cc:1514] Processing DeleteTablet for tablet 8656560a80ca4b789e1e2f0e23efc224 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 15:01:53 UTC) from {username='kudu', principal='kudu/127.13.75.190@KRBTEST.COM'} at 127.0.0.1:40534
I20260923 15:01:53.150513 18086 tablet_service.cc:1514] Processing DeleteTablet for tablet 8656560a80ca4b789e1e2f0e23efc224 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 15:01:53 UTC) from {username='kudu', principal='kudu/127.13.75.190@KRBTEST.COM'} at 127.0.0.1:55544
I20260923 15:01:53.150499 17953 tablet_service.cc:1514] Processing DeleteTablet for tablet 8656560a80ca4b789e1e2f0e23efc224 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 15:01:53 UTC) from {username='kudu', principal='kudu/127.13.75.190@KRBTEST.COM'} at 127.0.0.1:45792
I20260923 15:01:53.150743 18193 tablet_replica.cc:330] T 8656560a80ca4b789e1e2f0e23efc224 P 8a2fbb1eaff540f1bc125b03da5a7fa1: stopping tablet replica
I20260923 15:01:53.150846 18193 raft_consensus.cc:2229] T 8656560a80ca4b789e1e2f0e23efc224 P 8a2fbb1eaff540f1bc125b03da5a7fa1 [term 1 LEADER]: Raft consensus shutting down.
I20260923 15:01:53.150991 18193 raft_consensus.cc:2258] T 8656560a80ca4b789e1e2f0e23efc224 P 8a2fbb1eaff540f1bc125b03da5a7fa1 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 15:01:53.151260 18194 tablet_replica.cc:330] T 8656560a80ca4b789e1e2f0e23efc224 P 954ac86dadd5427a91ced035a4d099b4: stopping tablet replica
I20260923 15:01:53.151261 13614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKC4soy/build/release/bin/kudu with pid 17758
I20260923 15:01:53.151360 18194 raft_consensus.cc:2229] T 8656560a80ca4b789e1e2f0e23efc224 P 954ac86dadd5427a91ced035a4d099b4 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 15:01:53.151463 18194 raft_consensus.cc:2258] T 8656560a80ca4b789e1e2f0e23efc224 P 954ac86dadd5427a91ced035a4d099b4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 15:01:53.151964 18194 ts_tablet_manager.cc:1905] T 8656560a80ca4b789e1e2f0e23efc224 P 954ac86dadd5427a91ced035a4d099b4: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 15:01:53.152371 18193 ts_tablet_manager.cc:1905] T 8656560a80ca4b789e1e2f0e23efc224 P 8a2fbb1eaff540f1bc125b03da5a7fa1: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 15:01:53.153966 18194 ts_tablet_manager.cc:1918] T 8656560a80ca4b789e1e2f0e23efc224 P 954ac86dadd5427a91ced035a4d099b4: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 15:01:53.154048 18194 log.cc:1196] T 8656560a80ca4b789e1e2f0e23efc224 P 954ac86dadd5427a91ced035a4d099b4: Deleting WAL directory at /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175691491353-13614-0/minicluster-data/ts-2/wal/wals/8656560a80ca4b789e1e2f0e23efc224
I20260923 15:01:53.154328 18194 ts_tablet_manager.cc:1939] T 8656560a80ca4b789e1e2f0e23efc224 P 954ac86dadd5427a91ced035a4d099b4: Deleting consensus metadata
I20260923 15:01:53.154762 17683 catalog_manager.cc:4841] TS 954ac86dadd5427a91ced035a4d099b4 (127.13.75.131:43899): tablet 8656560a80ca4b789e1e2f0e23efc224 (table test-table [id=c8e5af0becd64a82b552363fd5c27762]) successfully deleted
I20260923 15:01:53.155252 18193 ts_tablet_manager.cc:1918] T 8656560a80ca4b789e1e2f0e23efc224 P 8a2fbb1eaff540f1bc125b03da5a7fa1: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 15:01:53.155395 18193 log.cc:1196] T 8656560a80ca4b789e1e2f0e23efc224 P 8a2fbb1eaff540f1bc125b03da5a7fa1: Deleting WAL directory at /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestDisableWebUI.1790175691491353-13614-0/minicluster-data/ts-1/wal/wals/8656560a80ca4b789e1e2f0e23efc224
I20260923 15:01:53.155751 18193 ts_tablet_manager.cc:1939] T 8656560a80ca4b789e1e2f0e23efc224 P 8a2fbb1eaff540f1bc125b03da5a7fa1: Deleting consensus metadata
I20260923 15:01:53.156390 17682 catalog_manager.cc:4841] TS 8a2fbb1eaff540f1bc125b03da5a7fa1 (127.13.75.130:45909): tablet 8656560a80ca4b789e1e2f0e23efc224 (table test-table [id=c8e5af0becd64a82b552363fd5c27762]) successfully deleted
W20260923 15:01:53.157989 17683 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.75.129:44285 (error 108)
I20260923 15:01:53.158349 13614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKC4soy/build/release/bin/kudu with pid 17891
I20260923 15:01:53.159044 18163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:53.158435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.129:44285 (local address 127.0.0.1:40550)
0923 15:01:53.158940 (+ 505us) negotiation.cc:107] Waiting for socket to connect
0923 15:01:53.158997 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.129:44285: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":472}
W20260923 15:01:53.163036 17683 catalog_manager.cc:4584] TS 92215cf8e3704829aaec7d52d2d8f191 (127.13.75.129:44285): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 8656560a80ca4b789e1e2f0e23efc224: Network error: Client connection negotiation failed: client connection to 127.13.75.129:44285: connect: Connection refused (error 111)
I20260923 15:01:53.163149 13614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKC4soy/build/release/bin/kudu with pid 18024
I20260923 15:01:53.168123 13614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKC4soy/build/release/bin/kudu with pid 17665
2026-09-23T15:01:53Z chronyd exiting
[ OK ] SecurityITest.TestDisableWebUI (2058 ms)
[ RUN ] SecurityITest.TestDisableAuthenticationEncryption
2026-09-23T15:01:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T15:01:53Z Disabled control of system clock
I20260923 15:01:53.197219 13614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKC4soy/build/release/bin/kudu
/tmp/dist-test-taskKC4soy/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790175691491353-13614-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790175691491353-13614-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790175691491353-13614-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790175691491353-13614-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.75.190:35381
--webserver_interface=127.13.75.190
--webserver_port=0
--builtin_ntp_servers=127.13.75.148:45427
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.75.190:35381
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260923 15:01:53.280385 18199 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:01:53.280630 18199 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:01:53.280655 18199 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:01:53.282186 18199 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 15:01:53.282238 18199 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:01:53.282253 18199 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:01:53.282265 18199 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 15:01:53.282280 18199 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 15:01:53.283826 18199 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.75.148:45427
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790175691491353-13614-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790175691491353-13614-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.75.190:35381
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.75.190:35381
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790175691491353-13614-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.75.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18199
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790175691491353-13614-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:01:53.284150 18199 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:01:53.284482 18199 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260923 15:01:53.290356 18199 server_base.cc:1034] running on GCE node
W20260923 15:01:53.290318 18204 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:01:53.290323 18207 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:01:53.290318 18205 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:01:53.291165 18199 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:01:53.291522 18199 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:01:53.292682 18199 hybrid_clock.cc:648] HybridClock initialized: now 1790175713292603 us; error 103 us; skew 500 ppm
I20260923 15:01:53.294090 18199 webserver.cc:460] Webserver started at http://127.13.75.190:38323/ using document root <none> and password file <none>
I20260923 15:01:53.294366 18199 fs_manager.cc:359] Metadata directory not provided
I20260923 15:01:53.294418 18199 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:01:53.294529 18199 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:01:53.295744 18199 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790175691491353-13614-0/minicluster-data/master-0/data/instance:
uuid: "59409d97492540269546e61dd12c121b"
format_stamp: "Formatted at 2026-09-23 15:01:53 on dist-test-slave-cnb0"
server_key: "fcd745e8b192ad3b6eb8140720719957"
server_key_iv: "650741a228cbe79fa53e94fa9213fe71"
server_key_version: "encryptionkey@0"
I20260923 15:01:53.296123 18199 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790175691491353-13614-0/minicluster-data/master-0/wal/instance:
uuid: "59409d97492540269546e61dd12c121b"
format_stamp: "Formatted at 2026-09-23 15:01:53 on dist-test-slave-cnb0"
server_key: "fcd745e8b192ad3b6eb8140720719957"
server_key_iv: "650741a228cbe79fa53e94fa9213fe71"
server_key_version: "encryptionkey@0"
I20260923 15:01:53.297848 18199 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260923 15:01:53.298871 18213 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:01:53.299192 18199 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260923 15:01:53.299305 18199 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790175691491353-13614-0/minicluster-data/master-0/data,/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790175691491353-13614-0/minicluster-data/master-0/wal
uuid: "59409d97492540269546e61dd12c121b"
format_stamp: "Formatted at 2026-09-23 15:01:53 on dist-test-slave-cnb0"
server_key: "fcd745e8b192ad3b6eb8140720719957"
server_key_iv: "650741a228cbe79fa53e94fa9213fe71"
server_key_version: "encryptionkey@0"
I20260923 15:01:53.299393 18199 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790175691491353-13614-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790175691491353-13614-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790175691491353-13614-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:01:53.303282 18199 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:01:53.303589 18199 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:01:53.303756 18199 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:01:53.307864 18199 rpc_server.cc:307] RPC server started. Bound to: 127.13.75.190:35381
I20260923 15:01:53.307895 18265 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.75.190:35381 every 8 connection(s)
I20260923 15:01:53.308285 18199 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790175691491353-13614-0/minicluster-data/master-0/data/info.pb
I20260923 15:01:53.309018 18266 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:01:53.311992 18266 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 59409d97492540269546e61dd12c121b: Bootstrap starting.
I20260923 15:01:53.312525 13614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKC4soy/build/release/bin/kudu as pid 18199
I20260923 15:01:53.312616 13614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790175691491353-13614-0/minicluster-data/master-0/wal/instance
I20260923 15:01:53.312680 18266 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 59409d97492540269546e61dd12c121b: Neither blocks nor log segments found. Creating new log.
I20260923 15:01:53.312839 13614 external_mini_cluster.cc:1428] Setting key d6fd6fc29bb8871144923e2d0a5bb37d
I20260923 15:01:53.313009 18266 log.cc:824] T 00000000000000000000000000000000 P 59409d97492540269546e61dd12c121b: Log is configured to *not* fsync() on all Append() calls
I20260923 15:01:53.313761 18266 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 59409d97492540269546e61dd12c121b: No bootstrap required, opened a new log
I20260923 15:01:53.315822 18270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:53.313459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33638 (local address 127.13.75.190:35381)
0923 15:01:53.313718 (+ 259us) server_negotiation.cc:206] Beginning negotiation
0923 15:01:53.313728 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0923 15:01:53.313768 (+ 40us) server_negotiation.cc:407] Connection header received
0923 15:01:53.314315 (+ 547us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:01:53.314323 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:01:53.314374 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:01:53.314552 (+ 178us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:01:53.315344 (+ 792us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:01:53.315348 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:01:53.315348 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 15:01:53.315353 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:01:53.315405 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 15:01:53.315418 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:01:53.315528 (+ 110us) server_negotiation.cc:1035] Waiting for connection context
0923 15:01:53.315571 (+ 43us) server_negotiation.cc:299] Negotiation successful
0923 15:01:53.315634 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":74,"threads_started":1}
I20260923 15:01:53.315953 18266 raft_consensus.cc:348] T 00000000000000000000000000000000 P 59409d97492540269546e61dd12c121b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59409d97492540269546e61dd12c121b" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 35381 } }
I20260923 15:01:53.316159 18266 raft_consensus.cc:374] T 00000000000000000000000000000000 P 59409d97492540269546e61dd12c121b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:01:53.316198 18266 raft_consensus.cc:729] T 00000000000000000000000000000000 P 59409d97492540269546e61dd12c121b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 59409d97492540269546e61dd12c121b, State: Initialized, Role: FOLLOWER
I20260923 15:01:53.316319 18266 consensus_queue.cc:260] T 00000000000000000000000000000000 P 59409d97492540269546e61dd12c121b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59409d97492540269546e61dd12c121b" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 35381 } }
I20260923 15:01:53.316406 18266 raft_consensus.cc:388] T 00000000000000000000000000000000 P 59409d97492540269546e61dd12c121b [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 15:01:53.316440 18266 raft_consensus.cc:482] T 00000000000000000000000000000000 P 59409d97492540269546e61dd12c121b [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 15:01:53.316511 18266 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 59409d97492540269546e61dd12c121b [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:01:53.317471 18266 raft_consensus.cc:504] T 00000000000000000000000000000000 P 59409d97492540269546e61dd12c121b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59409d97492540269546e61dd12c121b" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 35381 } }
I20260923 15:01:53.317562 18266 leader_election.cc:302] T 00000000000000000000000000000000 P 59409d97492540269546e61dd12c121b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 59409d97492540269546e61dd12c121b; no voters:
I20260923 15:01:53.317732 18266 leader_election.cc:288] T 00000000000000000000000000000000 P 59409d97492540269546e61dd12c121b [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 15:01:53.317832 18271 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 59409d97492540269546e61dd12c121b [term 1 FOLLOWER]: Leader election won for term 1
I20260923 15:01:53.318017 18271 raft_consensus.cc:686] T 00000000000000000000000000000000 P 59409d97492540269546e61dd12c121b [term 1 LEADER]: Becoming Leader. State: Replica: 59409d97492540269546e61dd12c121b, State: Running, Role: LEADER
I20260923 15:01:53.318017 18266 sys_catalog.cc:564] T 00000000000000000000000000000000 P 59409d97492540269546e61dd12c121b [sys.catalog]: configured and running, proceeding with master startup.
I20260923 15:01:53.318176 18271 consensus_queue.cc:237] T 00000000000000000000000000000000 P 59409d97492540269546e61dd12c121b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59409d97492540269546e61dd12c121b" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 35381 } }
I20260923 15:01:53.318581 18272 sys_catalog.cc:455] T 00000000000000000000000000000000 P 59409d97492540269546e61dd12c121b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "59409d97492540269546e61dd12c121b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59409d97492540269546e61dd12c121b" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 35381 } } }
I20260923 15:01:53.318607 18273 sys_catalog.cc:455] T 00000000000000000000000000000000 P 59409d97492540269546e61dd12c121b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 59409d97492540269546e61dd12c121b. Latest consensus state: current_term: 1 leader_uuid: "59409d97492540269546e61dd12c121b" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "59409d97492540269546e61dd12c121b" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 35381 } } }
I20260923 15:01:53.318707 18272 sys_catalog.cc:458] T 00000000000000000000000000000000 P 59409d97492540269546e61dd12c121b [sys.catalog]: This master's current role is: LEADER
I20260923 15:01:53.318719 18273 sys_catalog.cc:458] T 00000000000000000000000000000000 P 59409d97492540269546e61dd12c121b [sys.catalog]: This master's current role is: LEADER
I20260923 15:01:53.319018 18280 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 15:01:53.319604 18280 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 15:01:53.321328 18280 catalog_manager.cc:1347] Generated new cluster ID: 33475c51913f4b1fa7575abd42cb9249
I20260923 15:01:53.321386 18280 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 15:01:53.343783 18280 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 15:01:53.344041 18280 catalog_manager.cc:1504] Loading token signing keys...
I20260923 15:01:53.351132 18280 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 59409d97492540269546e61dd12c121b: Generated new TSK 0
I20260923 15:01:53.351521 18280 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 15:01:53.361593 13614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKC4soy/build/release/bin/kudu
/tmp/dist-test-taskKC4soy/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790175691491353-13614-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790175691491353-13614-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790175691491353-13614-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790175691491353-13614-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.75.129:0
--local_ip_for_outbound_sockets=127.13.75.129
--webserver_interface=127.13.75.129
--webserver_port=0
--tserver_master_addrs=127.13.75.190:35381
--builtin_ntp_servers=127.13.75.148:45427
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260923 15:01:53.455842 18290 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:01:53.456104 18290 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:01:53.456130 18290 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:01:53.457504 18290 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:01:53.457551 18290 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:01:53.457579 18290 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.75.129
I20260923 15:01:53.459003 18290 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.75.148:45427
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790175691491353-13614-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790175691491353-13614-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.75.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790175691491353-13614-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.75.129
--webserver_port=0
--tserver_master_addrs=127.13.75.190:35381
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18290
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.75.129
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790175691491353-13614-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:01:53.459270 18290 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:01:53.459599 18290 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:01:53.465027 18298 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:01:53.465036 18295 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:01:53.465041 18296 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:01:53.465704 18290 server_base.cc:1034] running on GCE node
I20260923 15:01:53.465962 18290 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:01:53.466270 18290 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:01:53.467458 18290 hybrid_clock.cc:648] HybridClock initialized: now 1790175713467407 us; error 59 us; skew 500 ppm
I20260923 15:01:53.469288 18290 webserver.cc:460] Webserver started at http://127.13.75.129:44463/ using document root <none> and password file <none>
I20260923 15:01:53.469580 18290 fs_manager.cc:359] Metadata directory not provided
I20260923 15:01:53.469651 18290 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:01:53.469790 18290 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:01:53.471462 18290 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790175691491353-13614-0/minicluster-data/ts-0/data/instance:
uuid: "8cbd05f7656b44aca30c3229acabf163"
format_stamp: "Formatted at 2026-09-23 15:01:53 on dist-test-slave-cnb0"
server_key: "befdf0a3119c22baac5fde60e6d15887"
server_key_iv: "d09a623e5909b47c96241907a9fb5cbf"
server_key_version: "encryptionkey@0"
I20260923 15:01:53.471931 18290 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790175691491353-13614-0/minicluster-data/ts-0/wal/instance:
uuid: "8cbd05f7656b44aca30c3229acabf163"
format_stamp: "Formatted at 2026-09-23 15:01:53 on dist-test-slave-cnb0"
server_key: "befdf0a3119c22baac5fde60e6d15887"
server_key_iv: "d09a623e5909b47c96241907a9fb5cbf"
server_key_version: "encryptionkey@0"
I20260923 15:01:53.473716 18290 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260923 15:01:53.474608 18304 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:01:53.474967 18290 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:01:53.475112 18290 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790175691491353-13614-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790175691491353-13614-0/minicluster-data/ts-0/wal
uuid: "8cbd05f7656b44aca30c3229acabf163"
format_stamp: "Formatted at 2026-09-23 15:01:53 on dist-test-slave-cnb0"
server_key: "befdf0a3119c22baac5fde60e6d15887"
server_key_iv: "d09a623e5909b47c96241907a9fb5cbf"
server_key_version: "encryptionkey@0"
I20260923 15:01:53.475219 18290 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790175691491353-13614-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790175691491353-13614-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790175691491353-13614-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:01:53.479110 18290 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:01:53.479427 18290 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:01:53.479594 18290 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:01:53.479838 18290 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 15:01:53.480192 18290 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 15:01:53.480255 18290 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 15:01:53.480301 18290 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 15:01:53.480330 18290 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 15:01:53.487001 18290 rpc_server.cc:307] RPC server started. Bound to: 127.13.75.129:44009
I20260923 15:01:53.487058 18417 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.75.129:44009 every 8 connection(s)
I20260923 15:01:53.487502 18290 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790175691491353-13614-0/minicluster-data/ts-0/data/info.pb
I20260923 15:01:53.487771 13614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKC4soy/build/release/bin/kudu as pid 18290
I20260923 15:01:53.487916 13614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790175691491353-13614-0/minicluster-data/ts-0/wal/instance
I20260923 15:01:53.488149 13614 external_mini_cluster.cc:1428] Setting key 94d7da893bb608908675f44accfb72ad
I20260923 15:01:53.489372 13614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKC4soy/build/release/bin/kudu
/tmp/dist-test-taskKC4soy/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790175691491353-13614-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790175691491353-13614-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790175691491353-13614-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790175691491353-13614-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.75.130:0
--local_ip_for_outbound_sockets=127.13.75.130
--webserver_interface=127.13.75.130
--webserver_port=0
--tserver_master_addrs=127.13.75.190:35381
--builtin_ntp_servers=127.13.75.148:45427
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
I20260923 15:01:53.491689 18270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:53.488483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.129:35275 (local address 127.13.75.190:35381)
0923 15:01:53.488588 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0923 15:01:53.488590 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:01:53.488870 (+ 280us) server_negotiation.cc:407] Connection header received
0923 15:01:53.489655 (+ 785us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:01:53.489656 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:01:53.489663 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:01:53.489695 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:01:53.491383 (+ 1688us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:01:53.491384 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:01:53.491385 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 15:01:53.491390 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:01:53.491407 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 15:01:53.491412 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:01:53.491502 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0923 15:01:53.491596 (+ 94us) server_negotiation.cc:299] Negotiation successful
0923 15:01:53.491613 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":78}
I20260923 15:01:53.491928 18420 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:53.488513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:35381 (local address 127.13.75.129:35275)
0923 15:01:53.488746 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0923 15:01:53.488783 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0923 15:01:53.489509 (+ 726us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:01:53.489910 (+ 401us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:01:53.489915 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:01:53.489925 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:01:53.491196 (+ 1271us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 15:01:53.491196 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:01:53.491229 (+ 33us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 15:01:53.491261 (+ 32us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:01:53.491502 (+ 241us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:01:53.491509 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:01:53.491513 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 15:01:53.491609 (+ 96us) client_negotiation.cc:240] Negotiation successful
0923 15:01:53.491679 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":105,"threads_started":1}
I20260923 15:01:53.492635 18418 heartbeater.cc:347] Connected to a master server at 127.13.75.190:35381
I20260923 15:01:53.492745 18418 heartbeater.cc:464] Registering TS with master...
I20260923 15:01:53.492918 18418 heartbeater.cc:511] Master 127.13.75.190:35381 requested a full tablet report, sending...
I20260923 15:01:53.493572 18230 ts_manager.cc:194] Registered new tserver with Master: 8cbd05f7656b44aca30c3229acabf163 (127.13.75.129:44009)
W20260923 15:01:53.578437 18421 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:01:53.578693 18421 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:01:53.578723 18421 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:01:53.580313 18421 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:01:53.580363 18421 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:01:53.580395 18421 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.75.130
I20260923 15:01:53.581979 18421 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.75.148:45427
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790175691491353-13614-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790175691491353-13614-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.75.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790175691491353-13614-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.75.130
--webserver_port=0
--tserver_master_addrs=127.13.75.190:35381
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18421
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.75.130
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790175691491353-13614-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:01:53.582293 18421 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:01:53.582646 18421 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:01:53.588943 18426 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:01:53.588901 18429 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:01:53.588905 18427 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:01:53.589339 18421 server_base.cc:1034] running on GCE node
I20260923 15:01:53.589690 18421 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:01:53.590109 18421 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:01:53.591298 18421 hybrid_clock.cc:648] HybridClock initialized: now 1790175713591275 us; error 41 us; skew 500 ppm
I20260923 15:01:53.592681 18421 webserver.cc:460] Webserver started at http://127.13.75.130:43241/ using document root <none> and password file <none>
I20260923 15:01:53.592964 18421 fs_manager.cc:359] Metadata directory not provided
I20260923 15:01:53.593040 18421 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:01:53.593197 18421 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:01:53.594287 18421 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790175691491353-13614-0/minicluster-data/ts-1/data/instance:
uuid: "e92c8ceea1474327ae1fe860a354c650"
format_stamp: "Formatted at 2026-09-23 15:01:53 on dist-test-slave-cnb0"
server_key: "0757317e5cb1fa14316c61e15b2fe390"
server_key_iv: "1b63ca4bec6fd5633ecd2de9066108ee"
server_key_version: "encryptionkey@0"
I20260923 15:01:53.594653 18421 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790175691491353-13614-0/minicluster-data/ts-1/wal/instance:
uuid: "e92c8ceea1474327ae1fe860a354c650"
format_stamp: "Formatted at 2026-09-23 15:01:53 on dist-test-slave-cnb0"
server_key: "0757317e5cb1fa14316c61e15b2fe390"
server_key_iv: "1b63ca4bec6fd5633ecd2de9066108ee"
server_key_version: "encryptionkey@0"
I20260923 15:01:53.596096 18421 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 15:01:53.596962 18435 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:01:53.597180 18421 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:01:53.597286 18421 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790175691491353-13614-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790175691491353-13614-0/minicluster-data/ts-1/wal
uuid: "e92c8ceea1474327ae1fe860a354c650"
format_stamp: "Formatted at 2026-09-23 15:01:53 on dist-test-slave-cnb0"
server_key: "0757317e5cb1fa14316c61e15b2fe390"
server_key_iv: "1b63ca4bec6fd5633ecd2de9066108ee"
server_key_version: "encryptionkey@0"
I20260923 15:01:53.597368 18421 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790175691491353-13614-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790175691491353-13614-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790175691491353-13614-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:01:53.600841 18421 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:01:53.601110 18421 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:01:53.601256 18421 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:01:53.601464 18421 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 15:01:53.601768 18421 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 15:01:53.601824 18421 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 15:01:53.601862 18421 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 15:01:53.601882 18421 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 15:01:53.607940 18421 rpc_server.cc:307] RPC server started. Bound to: 127.13.75.130:40541
I20260923 15:01:53.608012 18548 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.75.130:40541 every 8 connection(s)
I20260923 15:01:53.608342 18421 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790175691491353-13614-0/minicluster-data/ts-1/data/info.pb
I20260923 15:01:53.611574 18270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:53.609080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.130:53875 (local address 127.13.75.190:35381)
0923 15:01:53.609153 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0923 15:01:53.609155 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:01:53.609389 (+ 234us) server_negotiation.cc:407] Connection header received
0923 15:01:53.609964 (+ 575us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:01:53.609965 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:01:53.609972 (+ 7us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:01:53.610036 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:01:53.611280 (+ 1244us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:01:53.611282 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:01:53.611282 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 15:01:53.611287 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:01:53.611303 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 15:01:53.611308 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:01:53.611383 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0923 15:01:53.611484 (+ 101us) server_negotiation.cc:299] Negotiation successful
0923 15:01:53.611500 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260923 15:01:53.611797 18551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:53.609060 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:35381 (local address 127.13.75.130:53875)
0923 15:01:53.609265 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0923 15:01:53.609297 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0923 15:01:53.609837 (+ 540us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:01:53.610109 (+ 272us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:01:53.610113 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:01:53.610123 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:01:53.611091 (+ 968us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 15:01:53.611091 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:01:53.611120 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 15:01:53.611148 (+ 28us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:01:53.611381 (+ 233us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:01:53.611387 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:01:53.611390 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 15:01:53.611496 (+ 106us) client_negotiation.cc:240] Negotiation successful
0923 15:01:53.611559 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":89,"threads_started":1}
I20260923 15:01:53.612311 18549 heartbeater.cc:347] Connected to a master server at 127.13.75.190:35381
I20260923 15:01:53.612406 18549 heartbeater.cc:464] Registering TS with master...
I20260923 15:01:53.612576 18549 heartbeater.cc:511] Master 127.13.75.190:35381 requested a full tablet report, sending...
I20260923 15:01:53.613001 18230 ts_manager.cc:194] Registered new tserver with Master: e92c8ceea1474327ae1fe860a354c650 (127.13.75.130:40541)
I20260923 15:01:53.615926 13614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKC4soy/build/release/bin/kudu as pid 18421
I20260923 15:01:53.616009 13614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790175691491353-13614-0/minicluster-data/ts-1/wal/instance
I20260923 15:01:53.616217 13614 external_mini_cluster.cc:1428] Setting key 2d7d1b54769bd03e1b464bcb7105c9ba
I20260923 15:01:53.617290 13614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKC4soy/build/release/bin/kudu
/tmp/dist-test-taskKC4soy/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790175691491353-13614-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790175691491353-13614-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790175691491353-13614-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790175691491353-13614-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.75.131:0
--local_ip_for_outbound_sockets=127.13.75.131
--webserver_interface=127.13.75.131
--webserver_port=0
--tserver_master_addrs=127.13.75.190:35381
--builtin_ntp_servers=127.13.75.148:45427
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled
--rpc_encryption=disabled with env {}
W20260923 15:01:53.708103 18552 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:01:53.708366 18552 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:01:53.708416 18552 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:01:53.709702 18552 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:01:53.709784 18552 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:01:53.709828 18552 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.75.131
I20260923 15:01:53.711539 18552 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.75.148:45427
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790175691491353-13614-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790175691491353-13614-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.75.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790175691491353-13614-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.75.131
--webserver_port=0
--tserver_master_addrs=127.13.75.190:35381
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18552
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.75.131
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790175691491353-13614-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:01:53.711880 18552 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:01:53.712283 18552 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:01:53.717885 18560 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:01:53.717873 18558 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:01:53.718183 18552 server_base.cc:1034] running on GCE node
W20260923 15:01:53.717875 18557 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:01:53.718556 18552 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:01:53.718936 18552 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:01:53.720112 18552 hybrid_clock.cc:648] HybridClock initialized: now 1790175713720053 us; error 76 us; skew 500 ppm
I20260923 15:01:53.721618 18552 webserver.cc:460] Webserver started at http://127.13.75.131:35003/ using document root <none> and password file <none>
I20260923 15:01:53.721892 18552 fs_manager.cc:359] Metadata directory not provided
I20260923 15:01:53.721946 18552 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:01:53.722090 18552 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:01:53.723510 18552 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790175691491353-13614-0/minicluster-data/ts-2/data/instance:
uuid: "acae1130178d4dc49f9ea42702fbd930"
format_stamp: "Formatted at 2026-09-23 15:01:53 on dist-test-slave-cnb0"
server_key: "496138bfc12249c0bd4ea84eba909cab"
server_key_iv: "701bf89cc2ba9666d3afdbc6f363eb9f"
server_key_version: "encryptionkey@0"
I20260923 15:01:53.723954 18552 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790175691491353-13614-0/minicluster-data/ts-2/wal/instance:
uuid: "acae1130178d4dc49f9ea42702fbd930"
format_stamp: "Formatted at 2026-09-23 15:01:53 on dist-test-slave-cnb0"
server_key: "496138bfc12249c0bd4ea84eba909cab"
server_key_iv: "701bf89cc2ba9666d3afdbc6f363eb9f"
server_key_version: "encryptionkey@0"
I20260923 15:01:53.725448 18552 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 15:01:53.726459 18566 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:01:53.726812 18552 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:01:53.726989 18552 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790175691491353-13614-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790175691491353-13614-0/minicluster-data/ts-2/wal
uuid: "acae1130178d4dc49f9ea42702fbd930"
format_stamp: "Formatted at 2026-09-23 15:01:53 on dist-test-slave-cnb0"
server_key: "496138bfc12249c0bd4ea84eba909cab"
server_key_iv: "701bf89cc2ba9666d3afdbc6f363eb9f"
server_key_version: "encryptionkey@0"
I20260923 15:01:53.727101 18552 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790175691491353-13614-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790175691491353-13614-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790175691491353-13614-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:01:53.730865 18552 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:01:53.731213 18552 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:01:53.731364 18552 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:01:53.731603 18552 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 15:01:53.731945 18552 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 15:01:53.732002 18552 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 15:01:53.732041 18552 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 15:01:53.732067 18552 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 15:01:53.738260 18552 rpc_server.cc:307] RPC server started. Bound to: 127.13.75.131:41223
I20260923 15:01:53.738341 18679 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.75.131:41223 every 8 connection(s)
I20260923 15:01:53.738729 18552 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790175691491353-13614-0/minicluster-data/ts-2/data/info.pb
I20260923 15:01:53.741961 18270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:53.739607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.131:52413 (local address 127.13.75.190:35381)
0923 15:01:53.739695 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0923 15:01:53.739697 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:01:53.739895 (+ 198us) server_negotiation.cc:407] Connection header received
0923 15:01:53.740361 (+ 466us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:01:53.740362 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:01:53.740368 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:01:53.740430 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:01:53.741667 (+ 1237us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:01:53.741669 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:01:53.741669 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 15:01:53.741674 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:01:53.741689 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 15:01:53.741693 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:01:53.741762 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0923 15:01:53.741887 (+ 125us) server_negotiation.cc:299] Negotiation successful
0923 15:01:53.741901 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260923 15:01:53.742291 18682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:53.739514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:35381 (local address 127.13.75.131:52413)
0923 15:01:53.739740 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0923 15:01:53.739786 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0923 15:01:53.740272 (+ 486us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:01:53.740497 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:01:53.740501 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:01:53.740510 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:01:53.741439 (+ 929us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 15:01:53.741440 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:01:53.741478 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 15:01:53.741509 (+ 31us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:01:53.741786 (+ 277us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:01:53.741792 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:01:53.741796 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 15:01:53.741887 (+ 91us) client_negotiation.cc:240] Negotiation successful
0923 15:01:53.741943 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":90,"threads_started":1}
I20260923 15:01:53.742728 13614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKC4soy/build/release/bin/kudu as pid 18552
I20260923 15:01:53.742838 13614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790175691491353-13614-0/minicluster-data/ts-2/wal/instance
I20260923 15:01:53.742995 18680 heartbeater.cc:347] Connected to a master server at 127.13.75.190:35381
I20260923 15:01:53.743091 13614 external_mini_cluster.cc:1428] Setting key 634b1295eb0863ea9764826490bab681
I20260923 15:01:53.743098 18680 heartbeater.cc:464] Registering TS with master...
I20260923 15:01:53.743291 18680 heartbeater.cc:511] Master 127.13.75.190:35381 requested a full tablet report, sending...
I20260923 15:01:53.743891 18230 ts_manager.cc:194] Registered new tserver with Master: acae1130178d4dc49f9ea42702fbd930 (127.13.75.131:41223)
I20260923 15:01:53.744264 13614 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260923 15:01:53.747170 18270 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:53.745922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33646 (local address 127.13.75.190:35381)
0923 15:01:53.746001 (+ 79us) server_negotiation.cc:206] Beginning negotiation
0923 15:01:53.746003 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:01:53.746051 (+ 48us) server_negotiation.cc:407] Connection header received
0923 15:01:53.746111 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:01:53.746112 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:01:53.746117 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:01:53.746174 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:01:53.746875 (+ 701us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:01:53.746877 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:01:53.746955 (+ 78us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 15:01:53.746962 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:01:53.746985 (+ 23us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 15:01:53.746990 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:01:53.747047 (+ 57us) server_negotiation.cc:1035] Waiting for connection context
0923 15:01:53.747095 (+ 48us) server_negotiation.cc:299] Negotiation successful
0923 15:01:53.747107 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260923 15:01:53.749842 18230 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:33646:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 15:01:53.751803 18230 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 15:01:53.756296 18692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:53.754662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.131:41223 (local address 127.0.0.1:55170)
0923 15:01:53.754992 (+ 330us) negotiation.cc:107] Waiting for socket to connect
0923 15:01:53.755002 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 15:01:53.755050 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:01:53.755240 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:01:53.755246 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:01:53.755253 (+ 7us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:01:53.755825 (+ 572us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 15:01:53.755825 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:01:53.755840 (+ 15us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 15:01:53.755859 (+ 19us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:01:53.756117 (+ 258us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:01:53.756124 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:01:53.756127 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 15:01:53.756174 (+ 47us) client_negotiation.cc:240] Negotiation successful
0923 15:01:53.756187 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":312,"thread_start_us":87,"threads_started":1}
I20260923 15:01:53.756734 18691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:53.754755 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55170 (local address 127.13.75.131:41223)
0923 15:01:53.755011 (+ 256us) server_negotiation.cc:206] Beginning negotiation
0923 15:01:53.755016 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 15:01:53.755101 (+ 85us) server_negotiation.cc:407] Connection header received
0923 15:01:53.755132 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:01:53.755135 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:01:53.755151 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:01:53.755235 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:01:53.755943 (+ 708us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:01:53.755952 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:01:53.755952 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 15:01:53.755956 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:01:53.755999 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 15:01:53.756030 (+ 31us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:01:53.756584 (+ 554us) server_negotiation.cc:1035] Waiting for connection context
0923 15:01:53.756618 (+ 34us) server_negotiation.cc:299] Negotiation successful
0923 15:01:53.756637 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":211,"thread_start_us":87,"threads_started":1}
I20260923 15:01:53.756781 18689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:53.754850 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.130:40541 (local address 127.0.0.1:53958)
0923 15:01:53.755077 (+ 227us) negotiation.cc:107] Waiting for socket to connect
0923 15:01:53.755088 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 15:01:53.755137 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:01:53.755953 (+ 816us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:01:53.755954 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:01:53.755957 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:01:53.756368 (+ 411us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 15:01:53.756368 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:01:53.756376 (+ 8us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 15:01:53.756390 (+ 14us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:01:53.756633 (+ 243us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:01:53.756634 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:01:53.756635 (+ 1us) client_negotiation.cc:769] Sending connection context
0923 15:01:53.756681 (+ 46us) client_negotiation.cc:240] Negotiation successful
0923 15:01:53.756693 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":96,"threads_started":1}
I20260923 15:01:53.756882 18688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:53.754615 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.129:44009 (local address 127.0.0.1:58338)
0923 15:01:53.754830 (+ 215us) negotiation.cc:107] Waiting for socket to connect
0923 15:01:53.754878 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0923 15:01:53.754969 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:01:53.755633 (+ 664us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:01:53.755634 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:01:53.755637 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:01:53.756026 (+ 389us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 15:01:53.756027 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:01:53.756034 (+ 7us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 15:01:53.756049 (+ 15us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:01:53.756654 (+ 605us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:01:53.756655 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:01:53.756655 (+ 0us) client_negotiation.cc:769] Sending connection context
0923 15:01:53.756703 (+ 48us) client_negotiation.cc:240] Negotiation successful
0923 15:01:53.756719 (+ 16us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":182,"thread_start_us":87,"threads_started":1}
I20260923 15:01:53.756942 18690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:53.754704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58338 (local address 127.13.75.129:44009)
0923 15:01:53.755262 (+ 558us) server_negotiation.cc:206] Beginning negotiation
0923 15:01:53.755267 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 15:01:53.755276 (+ 9us) server_negotiation.cc:407] Connection header received
0923 15:01:53.755302 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:01:53.755310 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:01:53.755319 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:01:53.755413 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:01:53.756521 (+ 1108us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:01:53.756525 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:01:53.756525 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 15:01:53.756528 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:01:53.756559 (+ 31us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 15:01:53.756572 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:01:53.756627 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0923 15:01:53.756817 (+ 190us) server_negotiation.cc:299] Negotiation successful
0923 15:01:53.756835 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":517,"thread_start_us":81,"threads_started":1}
I20260923 15:01:53.756829 18693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:53.755107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53958 (local address 127.13.75.130:40541)
0923 15:01:53.755559 (+ 452us) server_negotiation.cc:206] Beginning negotiation
0923 15:01:53.755563 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 15:01:53.755577 (+ 14us) server_negotiation.cc:407] Connection header received
0923 15:01:53.755624 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:01:53.755626 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:01:53.755635 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:01:53.756116 (+ 481us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:01:53.756482 (+ 366us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:01:53.756491 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:01:53.756491 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 15:01:53.756496 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:01:53.756533 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 15:01:53.756561 (+ 28us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:01:53.756622 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0923 15:01:53.756703 (+ 81us) server_negotiation.cc:299] Negotiation successful
0923 15:01:53.756721 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":409,"thread_start_us":106,"threads_started":1}
I20260923 15:01:53.757673 18483 tablet_service.cc:1467] Processing CreateTablet for tablet 035b0ce432c34f6aac146ff9076f7a25 (DEFAULT_TABLE table=test-table [id=57c4a4cf79e2486c9575886648b53457]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 15:01:53.757725 18352 tablet_service.cc:1467] Processing CreateTablet for tablet 035b0ce432c34f6aac146ff9076f7a25 (DEFAULT_TABLE table=test-table [id=57c4a4cf79e2486c9575886648b53457]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 15:01:53.758047 18483 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 035b0ce432c34f6aac146ff9076f7a25. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:01:53.758054 18352 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 035b0ce432c34f6aac146ff9076f7a25. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:01:53.759855 18614 tablet_service.cc:1467] Processing CreateTablet for tablet 035b0ce432c34f6aac146ff9076f7a25 (DEFAULT_TABLE table=test-table [id=57c4a4cf79e2486c9575886648b53457]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 15:01:53.760215 18614 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 035b0ce432c34f6aac146ff9076f7a25. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:01:53.761302 18694 tablet_bootstrap.cc:492] T 035b0ce432c34f6aac146ff9076f7a25 P 8cbd05f7656b44aca30c3229acabf163: Bootstrap starting.
I20260923 15:01:53.761739 18695 tablet_bootstrap.cc:492] T 035b0ce432c34f6aac146ff9076f7a25 P e92c8ceea1474327ae1fe860a354c650: Bootstrap starting.
I20260923 15:01:53.762262 18694 tablet_bootstrap.cc:654] T 035b0ce432c34f6aac146ff9076f7a25 P 8cbd05f7656b44aca30c3229acabf163: Neither blocks nor log segments found. Creating new log.
I20260923 15:01:53.762377 18695 tablet_bootstrap.cc:654] T 035b0ce432c34f6aac146ff9076f7a25 P e92c8ceea1474327ae1fe860a354c650: Neither blocks nor log segments found. Creating new log.
I20260923 15:01:53.762629 18694 log.cc:824] T 035b0ce432c34f6aac146ff9076f7a25 P 8cbd05f7656b44aca30c3229acabf163: Log is configured to *not* fsync() on all Append() calls
I20260923 15:01:53.762678 18695 log.cc:824] T 035b0ce432c34f6aac146ff9076f7a25 P e92c8ceea1474327ae1fe860a354c650: Log is configured to *not* fsync() on all Append() calls
I20260923 15:01:53.763504 18694 tablet_bootstrap.cc:492] T 035b0ce432c34f6aac146ff9076f7a25 P 8cbd05f7656b44aca30c3229acabf163: No bootstrap required, opened a new log
I20260923 15:01:53.763526 18695 tablet_bootstrap.cc:492] T 035b0ce432c34f6aac146ff9076f7a25 P e92c8ceea1474327ae1fe860a354c650: No bootstrap required, opened a new log
I20260923 15:01:53.763577 18694 ts_tablet_manager.cc:1397] T 035b0ce432c34f6aac146ff9076f7a25 P 8cbd05f7656b44aca30c3229acabf163: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260923 15:01:53.763590 18695 ts_tablet_manager.cc:1397] T 035b0ce432c34f6aac146ff9076f7a25 P e92c8ceea1474327ae1fe860a354c650: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 15:01:53.763743 18698 tablet_bootstrap.cc:492] T 035b0ce432c34f6aac146ff9076f7a25 P acae1130178d4dc49f9ea42702fbd930: Bootstrap starting.
I20260923 15:01:53.764384 18698 tablet_bootstrap.cc:654] T 035b0ce432c34f6aac146ff9076f7a25 P acae1130178d4dc49f9ea42702fbd930: Neither blocks nor log segments found. Creating new log.
I20260923 15:01:53.764691 18698 log.cc:824] T 035b0ce432c34f6aac146ff9076f7a25 P acae1130178d4dc49f9ea42702fbd930: Log is configured to *not* fsync() on all Append() calls
I20260923 15:01:53.765574 18698 tablet_bootstrap.cc:492] T 035b0ce432c34f6aac146ff9076f7a25 P acae1130178d4dc49f9ea42702fbd930: No bootstrap required, opened a new log
I20260923 15:01:53.765640 18698 ts_tablet_manager.cc:1397] T 035b0ce432c34f6aac146ff9076f7a25 P acae1130178d4dc49f9ea42702fbd930: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 15:01:53.765607 18695 raft_consensus.cc:348] T 035b0ce432c34f6aac146ff9076f7a25 P e92c8ceea1474327ae1fe860a354c650 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cbd05f7656b44aca30c3229acabf163" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 44009 } } peers { permanent_uuid: "e92c8ceea1474327ae1fe860a354c650" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 40541 } } peers { permanent_uuid: "acae1130178d4dc49f9ea42702fbd930" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 41223 } }
I20260923 15:01:53.765729 18695 raft_consensus.cc:374] T 035b0ce432c34f6aac146ff9076f7a25 P e92c8ceea1474327ae1fe860a354c650 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:01:53.765750 18695 raft_consensus.cc:729] T 035b0ce432c34f6aac146ff9076f7a25 P e92c8ceea1474327ae1fe860a354c650 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e92c8ceea1474327ae1fe860a354c650, State: Initialized, Role: FOLLOWER
I20260923 15:01:53.765882 18695 consensus_queue.cc:260] T 035b0ce432c34f6aac146ff9076f7a25 P e92c8ceea1474327ae1fe860a354c650 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cbd05f7656b44aca30c3229acabf163" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 44009 } } peers { permanent_uuid: "e92c8ceea1474327ae1fe860a354c650" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 40541 } } peers { permanent_uuid: "acae1130178d4dc49f9ea42702fbd930" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 41223 } }
I20260923 15:01:53.766150 18695 ts_tablet_manager.cc:1428] T 035b0ce432c34f6aac146ff9076f7a25 P e92c8ceea1474327ae1fe860a354c650: Time spent starting tablet: real 0.003s user 0.000s sys 0.002s
I20260923 15:01:53.766170 18549 heartbeater.cc:503] Master 127.13.75.190:35381 was elected leader, sending a full tablet report...
I20260923 15:01:53.766283 18694 raft_consensus.cc:348] T 035b0ce432c34f6aac146ff9076f7a25 P 8cbd05f7656b44aca30c3229acabf163 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cbd05f7656b44aca30c3229acabf163" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 44009 } } peers { permanent_uuid: "e92c8ceea1474327ae1fe860a354c650" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 40541 } } peers { permanent_uuid: "acae1130178d4dc49f9ea42702fbd930" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 41223 } }
I20260923 15:01:53.766423 18694 raft_consensus.cc:374] T 035b0ce432c34f6aac146ff9076f7a25 P 8cbd05f7656b44aca30c3229acabf163 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:01:53.766458 18694 raft_consensus.cc:729] T 035b0ce432c34f6aac146ff9076f7a25 P 8cbd05f7656b44aca30c3229acabf163 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8cbd05f7656b44aca30c3229acabf163, State: Initialized, Role: FOLLOWER
I20260923 15:01:53.766590 18694 consensus_queue.cc:260] T 035b0ce432c34f6aac146ff9076f7a25 P 8cbd05f7656b44aca30c3229acabf163 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cbd05f7656b44aca30c3229acabf163" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 44009 } } peers { permanent_uuid: "e92c8ceea1474327ae1fe860a354c650" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 40541 } } peers { permanent_uuid: "acae1130178d4dc49f9ea42702fbd930" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 41223 } }
I20260923 15:01:53.766844 18694 ts_tablet_manager.cc:1428] T 035b0ce432c34f6aac146ff9076f7a25 P 8cbd05f7656b44aca30c3229acabf163: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260923 15:01:53.766981 18700 raft_consensus.cc:482] T 035b0ce432c34f6aac146ff9076f7a25 P e92c8ceea1474327ae1fe860a354c650 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 15:01:53.766999 18418 heartbeater.cc:503] Master 127.13.75.190:35381 was elected leader, sending a full tablet report...
I20260923 15:01:53.767055 18700 raft_consensus.cc:504] T 035b0ce432c34f6aac146ff9076f7a25 P e92c8ceea1474327ae1fe860a354c650 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cbd05f7656b44aca30c3229acabf163" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 44009 } } peers { permanent_uuid: "e92c8ceea1474327ae1fe860a354c650" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 40541 } } peers { permanent_uuid: "acae1130178d4dc49f9ea42702fbd930" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 41223 } }
I20260923 15:01:53.767652 18700 leader_election.cc:288] T 035b0ce432c34f6aac146ff9076f7a25 P e92c8ceea1474327ae1fe860a354c650 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 8cbd05f7656b44aca30c3229acabf163 (127.13.75.129:44009), acae1130178d4dc49f9ea42702fbd930 (127.13.75.131:41223)
I20260923 15:01:53.768577 18690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:53.767589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.130:58107 (local address 127.13.75.129:44009)
0923 15:01:53.767694 (+ 105us) server_negotiation.cc:206] Beginning negotiation
0923 15:01:53.767696 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:01:53.767704 (+ 8us) server_negotiation.cc:407] Connection header received
0923 15:01:53.767723 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:01:53.767724 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:01:53.767728 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:01:53.767788 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:01:53.768369 (+ 581us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:01:53.768371 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:01:53.768371 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 15:01:53.768376 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:01:53.768391 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 15:01:53.768394 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:01:53.768452 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0923 15:01:53.768511 (+ 59us) server_negotiation.cc:299] Negotiation successful
0923 15:01:53.768524 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":84}
I20260923 15:01:53.768575 18551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:53.767468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.129:44009 (local address 127.13.75.130:58107)
0923 15:01:53.767527 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0923 15:01:53.767538 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 15:01:53.767572 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:01:53.767794 (+ 222us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:01:53.767795 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:01:53.767798 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:01:53.768265 (+ 467us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 15:01:53.768266 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:01:53.768273 (+ 7us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 15:01:53.768284 (+ 11us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:01:53.768451 (+ 167us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:01:53.768452 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:01:53.768453 (+ 1us) client_negotiation.cc:769] Sending connection context
0923 15:01:53.768495 (+ 42us) client_negotiation.cc:240] Negotiation successful
0923 15:01:53.768510 (+ 15us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260923 15:01:53.769134 18372 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "035b0ce432c34f6aac146ff9076f7a25" candidate_uuid: "e92c8ceea1474327ae1fe860a354c650" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8cbd05f7656b44aca30c3229acabf163" is_pre_election: true
I20260923 15:01:53.769266 18372 raft_consensus.cc:2454] T 035b0ce432c34f6aac146ff9076f7a25 P 8cbd05f7656b44aca30c3229acabf163 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e92c8ceea1474327ae1fe860a354c650 in term 0.
I20260923 15:01:53.769469 18436 leader_election.cc:302] T 035b0ce432c34f6aac146ff9076f7a25 P e92c8ceea1474327ae1fe860a354c650 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8cbd05f7656b44aca30c3229acabf163, e92c8ceea1474327ae1fe860a354c650; no voters:
I20260923 15:01:53.769605 18700 raft_consensus.cc:2781] T 035b0ce432c34f6aac146ff9076f7a25 P e92c8ceea1474327ae1fe860a354c650 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 15:01:53.769518 18698 raft_consensus.cc:348] T 035b0ce432c34f6aac146ff9076f7a25 P acae1130178d4dc49f9ea42702fbd930 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cbd05f7656b44aca30c3229acabf163" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 44009 } } peers { permanent_uuid: "e92c8ceea1474327ae1fe860a354c650" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 40541 } } peers { permanent_uuid: "acae1130178d4dc49f9ea42702fbd930" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 41223 } }
I20260923 15:01:53.769656 18700 raft_consensus.cc:482] T 035b0ce432c34f6aac146ff9076f7a25 P e92c8ceea1474327ae1fe860a354c650 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 15:01:53.769675 18698 raft_consensus.cc:374] T 035b0ce432c34f6aac146ff9076f7a25 P acae1130178d4dc49f9ea42702fbd930 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:01:53.769692 18700 raft_consensus.cc:3037] T 035b0ce432c34f6aac146ff9076f7a25 P e92c8ceea1474327ae1fe860a354c650 [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:01:53.769708 18698 raft_consensus.cc:729] T 035b0ce432c34f6aac146ff9076f7a25 P acae1130178d4dc49f9ea42702fbd930 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: acae1130178d4dc49f9ea42702fbd930, State: Initialized, Role: FOLLOWER
I20260923 15:01:53.769817 18691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:53.768330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.130:42583 (local address 127.13.75.131:41223)
0923 15:01:53.768381 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0923 15:01:53.768383 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:01:53.768720 (+ 337us) server_negotiation.cc:407] Connection header received
0923 15:01:53.768750 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:01:53.768750 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:01:53.768755 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:01:53.768823 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:01:53.769463 (+ 640us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:01:53.769464 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:01:53.769464 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 15:01:53.769469 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:01:53.769482 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 15:01:53.769486 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:01:53.769541 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0923 15:01:53.769690 (+ 149us) server_negotiation.cc:299] Negotiation successful
0923 15:01:53.769701 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260923 15:01:53.769860 18698 consensus_queue.cc:260] T 035b0ce432c34f6aac146ff9076f7a25 P acae1130178d4dc49f9ea42702fbd930 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cbd05f7656b44aca30c3229acabf163" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 44009 } } peers { permanent_uuid: "e92c8ceea1474327ae1fe860a354c650" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 40541 } } peers { permanent_uuid: "acae1130178d4dc49f9ea42702fbd930" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 41223 } }
I20260923 15:01:53.770157 18698 ts_tablet_manager.cc:1428] T 035b0ce432c34f6aac146ff9076f7a25 P acae1130178d4dc49f9ea42702fbd930: Time spent starting tablet: real 0.004s user 0.004s sys 0.000s
I20260923 15:01:53.770188 18680 heartbeater.cc:503] Master 127.13.75.190:35381 was elected leader, sending a full tablet report...
I20260923 15:01:53.770725 18700 raft_consensus.cc:504] T 035b0ce432c34f6aac146ff9076f7a25 P e92c8ceea1474327ae1fe860a354c650 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cbd05f7656b44aca30c3229acabf163" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 44009 } } peers { permanent_uuid: "e92c8ceea1474327ae1fe860a354c650" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 40541 } } peers { permanent_uuid: "acae1130178d4dc49f9ea42702fbd930" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 41223 } }
I20260923 15:01:53.770867 18700 leader_election.cc:288] T 035b0ce432c34f6aac146ff9076f7a25 P e92c8ceea1474327ae1fe860a354c650 [CANDIDATE]: Term 1 election: Requested vote from peers 8cbd05f7656b44aca30c3229acabf163 (127.13.75.129:44009), acae1130178d4dc49f9ea42702fbd930 (127.13.75.131:41223)
I20260923 15:01:53.771056 18551 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:53.768395 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.131:41223 (local address 127.13.75.130:42583)
0923 15:01:53.768640 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0923 15:01:53.768652 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 15:01:53.768695 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:01:53.768894 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:01:53.768895 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:01:53.768898 (+ 3us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:01:53.769363 (+ 465us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 15:01:53.769364 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:01:53.769370 (+ 6us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 15:01:53.769381 (+ 11us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:01:53.769561 (+ 180us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:01:53.769561 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:01:53.769562 (+ 1us) client_negotiation.cc:769] Sending connection context
0923 15:01:53.769616 (+ 54us) client_negotiation.cc:240] Negotiation successful
0923 15:01:53.769629 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":87,"threads_started":1}
I20260923 15:01:53.771113 18372 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "035b0ce432c34f6aac146ff9076f7a25" candidate_uuid: "e92c8ceea1474327ae1fe860a354c650" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8cbd05f7656b44aca30c3229acabf163"
I20260923 15:01:53.771184 18372 raft_consensus.cc:3037] T 035b0ce432c34f6aac146ff9076f7a25 P 8cbd05f7656b44aca30c3229acabf163 [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:01:53.771417 18634 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "035b0ce432c34f6aac146ff9076f7a25" candidate_uuid: "e92c8ceea1474327ae1fe860a354c650" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acae1130178d4dc49f9ea42702fbd930" is_pre_election: true
I20260923 15:01:53.771566 18634 raft_consensus.cc:2454] T 035b0ce432c34f6aac146ff9076f7a25 P acae1130178d4dc49f9ea42702fbd930 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e92c8ceea1474327ae1fe860a354c650 in term 0.
I20260923 15:01:53.771695 18633 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "035b0ce432c34f6aac146ff9076f7a25" candidate_uuid: "e92c8ceea1474327ae1fe860a354c650" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "acae1130178d4dc49f9ea42702fbd930"
I20260923 15:01:53.771757 18633 raft_consensus.cc:3037] T 035b0ce432c34f6aac146ff9076f7a25 P acae1130178d4dc49f9ea42702fbd930 [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:01:53.772096 18372 raft_consensus.cc:2454] T 035b0ce432c34f6aac146ff9076f7a25 P 8cbd05f7656b44aca30c3229acabf163 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e92c8ceea1474327ae1fe860a354c650 in term 1.
I20260923 15:01:53.772292 18436 leader_election.cc:302] T 035b0ce432c34f6aac146ff9076f7a25 P e92c8ceea1474327ae1fe860a354c650 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 8cbd05f7656b44aca30c3229acabf163, e92c8ceea1474327ae1fe860a354c650; no voters:
I20260923 15:01:53.772418 18700 raft_consensus.cc:2781] T 035b0ce432c34f6aac146ff9076f7a25 P e92c8ceea1474327ae1fe860a354c650 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 15:01:53.772519 18633 raft_consensus.cc:2454] T 035b0ce432c34f6aac146ff9076f7a25 P acae1130178d4dc49f9ea42702fbd930 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e92c8ceea1474327ae1fe860a354c650 in term 1.
I20260923 15:01:53.772606 18700 raft_consensus.cc:686] T 035b0ce432c34f6aac146ff9076f7a25 P e92c8ceea1474327ae1fe860a354c650 [term 1 LEADER]: Becoming Leader. State: Replica: e92c8ceea1474327ae1fe860a354c650, State: Running, Role: LEADER
I20260923 15:01:53.772707 18700 consensus_queue.cc:237] T 035b0ce432c34f6aac146ff9076f7a25 P e92c8ceea1474327ae1fe860a354c650 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cbd05f7656b44aca30c3229acabf163" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 44009 } } peers { permanent_uuid: "e92c8ceea1474327ae1fe860a354c650" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 40541 } } peers { permanent_uuid: "acae1130178d4dc49f9ea42702fbd930" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 41223 } }
I20260923 15:01:53.773541 18230 catalog_manager.cc:5495] T 035b0ce432c34f6aac146ff9076f7a25 P e92c8ceea1474327ae1fe860a354c650 reported cstate change: term changed from 0 to 1, leader changed from <none> to e92c8ceea1474327ae1fe860a354c650 (127.13.75.130). New cstate: current_term: 1 leader_uuid: "e92c8ceea1474327ae1fe860a354c650" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "8cbd05f7656b44aca30c3229acabf163" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 44009 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e92c8ceea1474327ae1fe860a354c650" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 40541 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "acae1130178d4dc49f9ea42702fbd930" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 41223 } health_report { overall_health: UNKNOWN } } }
I20260923 15:01:53.781467 18693 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:53.780179 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53960 (local address 127.13.75.130:40541)
0923 15:01:53.780250 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0923 15:01:53.780252 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:01:53.780262 (+ 10us) server_negotiation.cc:407] Connection header received
0923 15:01:53.780332 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:01:53.780333 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:01:53.780350 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:01:53.780452 (+ 102us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:01:53.781128 (+ 676us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:01:53.781130 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:01:53.781130 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 15:01:53.781136 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:01:53.781153 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 15:01:53.781158 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:01:53.781218 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0923 15:01:53.781352 (+ 134us) server_negotiation.cc:299] Negotiation successful
0923 15:01:53.781370 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260923 15:01:53.783982 18633 raft_consensus.cc:1261] T 035b0ce432c34f6aac146ff9076f7a25 P acae1130178d4dc49f9ea42702fbd930 [term 1 FOLLOWER]: Refusing update from remote peer e92c8ceea1474327ae1fe860a354c650: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 15:01:53.783990 18372 raft_consensus.cc:1261] T 035b0ce432c34f6aac146ff9076f7a25 P 8cbd05f7656b44aca30c3229acabf163 [term 1 FOLLOWER]: Refusing update from remote peer e92c8ceea1474327ae1fe860a354c650: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 15:01:53.784384 18700 consensus_queue.cc:1035] T 035b0ce432c34f6aac146ff9076f7a25 P e92c8ceea1474327ae1fe860a354c650 [LEADER]: Connected to new peer: Peer: permanent_uuid: "acae1130178d4dc49f9ea42702fbd930" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 41223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 15:01:53.784590 18704 consensus_queue.cc:1035] T 035b0ce432c34f6aac146ff9076f7a25 P e92c8ceea1474327ae1fe860a354c650 [LEADER]: Connected to new peer: Peer: permanent_uuid: "8cbd05f7656b44aca30c3229acabf163" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 44009 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 15:01:53.787334 18709 mvcc.cc:205] Tried to move back new op lower bound from 7332559723655057408 to 7332559723613843456. Current Snapshot: MvccSnapshot[applied={T|T < 7332559723655057408}]
I20260923 15:01:53.787761 18708 mvcc.cc:205] Tried to move back new op lower bound from 7332559723655057408 to 7332559723613843456. Current Snapshot: MvccSnapshot[applied={T|T < 7332559723655057408}]
I20260923 15:01:53.789758 18229 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:33646:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 15:01:53.789850 18229 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:33646:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 15:01:53.790676 18229 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 59409d97492540269546e61dd12c121b: Sending DeleteTablet for 3 replicas of tablet 035b0ce432c34f6aac146ff9076f7a25
I20260923 15:01:53.791085 18614 tablet_service.cc:1514] Processing DeleteTablet for tablet 035b0ce432c34f6aac146ff9076f7a25 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 15:01:53 UTC) from {username='slave'} at 127.0.0.1:55170
I20260923 15:01:53.791054 18352 tablet_service.cc:1514] Processing DeleteTablet for tablet 035b0ce432c34f6aac146ff9076f7a25 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 15:01:53 UTC) from {username='slave'} at 127.0.0.1:58338
I20260923 15:01:53.791052 18483 tablet_service.cc:1514] Processing DeleteTablet for tablet 035b0ce432c34f6aac146ff9076f7a25 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 15:01:53 UTC) from {username='slave'} at 127.0.0.1:53958
I20260923 15:01:53.791313 18716 tablet_replica.cc:330] T 035b0ce432c34f6aac146ff9076f7a25 P acae1130178d4dc49f9ea42702fbd930: stopping tablet replica
I20260923 15:01:53.791419 18716 raft_consensus.cc:2229] T 035b0ce432c34f6aac146ff9076f7a25 P acae1130178d4dc49f9ea42702fbd930 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 15:01:53.791538 18716 raft_consensus.cc:2258] T 035b0ce432c34f6aac146ff9076f7a25 P acae1130178d4dc49f9ea42702fbd930 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 15:01:53.791600 13614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKC4soy/build/release/bin/kudu with pid 18290
I20260923 15:01:53.794523 18716 ts_tablet_manager.cc:1905] T 035b0ce432c34f6aac146ff9076f7a25 P acae1130178d4dc49f9ea42702fbd930: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 15:01:53.796350 18716 ts_tablet_manager.cc:1918] T 035b0ce432c34f6aac146ff9076f7a25 P acae1130178d4dc49f9ea42702fbd930: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 15:01:53.796422 18716 log.cc:1196] T 035b0ce432c34f6aac146ff9076f7a25 P acae1130178d4dc49f9ea42702fbd930: Deleting WAL directory at /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790175691491353-13614-0/minicluster-data/ts-2/wal/wals/035b0ce432c34f6aac146ff9076f7a25
I20260923 15:01:53.796692 18716 ts_tablet_manager.cc:1939] T 035b0ce432c34f6aac146ff9076f7a25 P acae1130178d4dc49f9ea42702fbd930: Deleting consensus metadata
I20260923 15:01:53.797169 18215 catalog_manager.cc:4841] TS acae1130178d4dc49f9ea42702fbd930 (127.13.75.131:41223): tablet 035b0ce432c34f6aac146ff9076f7a25 (table test-table [id=57c4a4cf79e2486c9575886648b53457]) successfully deleted
I20260923 15:01:53.799012 18717 tablet_replica.cc:330] T 035b0ce432c34f6aac146ff9076f7a25 P e92c8ceea1474327ae1fe860a354c650: stopping tablet replica
I20260923 15:01:53.799158 18717 raft_consensus.cc:2229] T 035b0ce432c34f6aac146ff9076f7a25 P e92c8ceea1474327ae1fe860a354c650 [term 1 LEADER]: Raft consensus shutting down.
I20260923 15:01:53.799325 18717 raft_consensus.cc:2258] T 035b0ce432c34f6aac146ff9076f7a25 P e92c8ceea1474327ae1fe860a354c650 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 15:01:53.799813 18717 ts_tablet_manager.cc:1905] T 035b0ce432c34f6aac146ff9076f7a25 P e92c8ceea1474327ae1fe860a354c650: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 15:01:53.801404 18717 ts_tablet_manager.cc:1918] T 035b0ce432c34f6aac146ff9076f7a25 P e92c8ceea1474327ae1fe860a354c650: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 15:01:53.801496 18717 log.cc:1196] T 035b0ce432c34f6aac146ff9076f7a25 P e92c8ceea1474327ae1fe860a354c650: Deleting WAL directory at /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestDisableAuthenticationEncryption.1790175691491353-13614-0/minicluster-data/ts-1/wal/wals/035b0ce432c34f6aac146ff9076f7a25
I20260923 15:01:53.801728 18717 ts_tablet_manager.cc:1939] T 035b0ce432c34f6aac146ff9076f7a25 P e92c8ceea1474327ae1fe860a354c650: Deleting consensus metadata
I20260923 15:01:53.802260 18214 catalog_manager.cc:4841] TS e92c8ceea1474327ae1fe860a354c650 (127.13.75.130:40541): tablet 035b0ce432c34f6aac146ff9076f7a25 (table test-table [id=57c4a4cf79e2486c9575886648b53457]) successfully deleted
W20260923 15:01:53.803306 18214 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.75.129:44009 (error 108)
I20260923 15:01:53.803464 13614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKC4soy/build/release/bin/kudu with pid 18421
I20260923 15:01:53.804591 18688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:53.804403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.129:44009 (local address 127.0.0.1:58346)
0923 15:01:53.804490 (+ 87us) negotiation.cc:107] Waiting for socket to connect
0923 15:01:53.804548 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.129:44009: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":62}
W20260923 15:01:53.804713 18214 catalog_manager.cc:4584] TS 8cbd05f7656b44aca30c3229acabf163 (127.13.75.129:44009): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 035b0ce432c34f6aac146ff9076f7a25: Network error: Client connection negotiation failed: client connection to 127.13.75.129:44009: connect: Connection refused (error 111)
I20260923 15:01:53.808794 13614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKC4soy/build/release/bin/kudu with pid 18552
I20260923 15:01:53.813377 13614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKC4soy/build/release/bin/kudu with pid 18199
2026-09-23T15:01:53Z chronyd exiting
[ OK ] SecurityITest.TestDisableAuthenticationEncryption (644 ms)
[ RUN ] SecurityITest.TestJwtMiniCluster
I20260923 15:01:53.833818 13614 mini_oidc.cc:152] Starting JWKS server
I20260923 15:01:53.835014 13614 webserver.cc:460] Webserver started at https://127.0.0.1:34773/ using document root <none> and password file <none>
I20260923 15:01:53.835114 13614 mini_oidc.cc:189] Starting OIDC Discovery server
I20260923 15:01:53.835306 13614 webserver.cc:460] Webserver started at http://127.0.0.1:38339/ using document root <none> and password file <none>
2026-09-23T15:01:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T15:01:53Z Disabled control of system clock
I20260923 15:01:53.843298 13614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKC4soy/build/release/bin/kudu
/tmp/dist-test-taskKC4soy/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790175691491353-13614-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790175691491353-13614-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790175691491353-13614-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790175691491353-13614-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.75.190:43211
--webserver_interface=127.13.75.190
--webserver_port=0
--builtin_ntp_servers=127.13.75.148:41311
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.75.190:43211
--enable_jwt_token_auth=true
--jwks_url=https://localhost:34773/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790175691491353-13614-0/testchainca.cert with env {}
W20260923 15:01:53.929299 18725 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:01:53.929513 18725 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:01:53.929538 18725 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:01:53.930665 18725 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 15:01:53.930699 18725 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:01:53.930712 18725 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:01:53.930724 18725 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 15:01:53.930737 18725 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260923 15:01:53.930749 18725 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260923 15:01:53.930760 18725 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:34773/jwks.json
I20260923 15:01:53.932436 18725 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.75.148:41311
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790175691491353-13614-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790175691491353-13614-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.75.190:43211
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.75.190:43211
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:34773/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790175691491353-13614-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.75.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790175691491353-13614-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18725
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790175691491353-13614-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:01:53.932652 18725 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:01:53.932821 18725 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:01:53.938103 18730 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:01:53.938084 18733 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:01:53.938076 18731 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:01:53.938292 18725 server_base.cc:1034] running on GCE node
I20260923 15:01:53.938544 18725 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:01:53.938858 18725 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:01:53.940096 18725 hybrid_clock.cc:648] HybridClock initialized: now 1790175713940079 us; error 26 us; skew 500 ppm
I20260923 15:01:53.941481 18725 webserver.cc:460] Webserver started at http://127.13.75.190:35311/ using document root <none> and password file <none>
I20260923 15:01:53.941816 18725 fs_manager.cc:359] Metadata directory not provided
I20260923 15:01:53.941864 18725 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:01:53.942041 18725 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:01:53.943240 18725 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790175691491353-13614-0/minicluster-data/master-0/data/instance:
uuid: "5547f1c032d348338ec2fc8fb921f4b3"
format_stamp: "Formatted at 2026-09-23 15:01:53 on dist-test-slave-cnb0"
server_key: "75171e0ece12ef74f9f78fc936636158"
server_key_iv: "06a44b3d526c082d469e4db73f2d340c"
server_key_version: "encryptionkey@0"
I20260923 15:01:53.943612 18725 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790175691491353-13614-0/minicluster-data/master-0/wal/instance:
uuid: "5547f1c032d348338ec2fc8fb921f4b3"
format_stamp: "Formatted at 2026-09-23 15:01:53 on dist-test-slave-cnb0"
server_key: "75171e0ece12ef74f9f78fc936636158"
server_key_iv: "06a44b3d526c082d469e4db73f2d340c"
server_key_version: "encryptionkey@0"
I20260923 15:01:53.944912 18725 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 15:01:53.945709 18739 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:01:53.945910 18725 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:01:53.946017 18725 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790175691491353-13614-0/minicluster-data/master-0/data,/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790175691491353-13614-0/minicluster-data/master-0/wal
uuid: "5547f1c032d348338ec2fc8fb921f4b3"
format_stamp: "Formatted at 2026-09-23 15:01:53 on dist-test-slave-cnb0"
server_key: "75171e0ece12ef74f9f78fc936636158"
server_key_iv: "06a44b3d526c082d469e4db73f2d340c"
server_key_version: "encryptionkey@0"
I20260923 15:01:53.946101 18725 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790175691491353-13614-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790175691491353-13614-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790175691491353-13614-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:01:53.967137 18725 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:01:53.967557 18725 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:01:53.967754 18725 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:01:53.972442 18725 rpc_server.cc:307] RPC server started. Bound to: 127.13.75.190:43211
I20260923 15:01:53.972481 18791 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.75.190:43211 every 8 connection(s)
I20260923 15:01:53.972882 18725 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790175691491353-13614-0/minicluster-data/master-0/data/info.pb
I20260923 15:01:53.973910 18792 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:01:53.976473 18792 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5547f1c032d348338ec2fc8fb921f4b3: Bootstrap starting.
I20260923 15:01:53.977123 18792 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 5547f1c032d348338ec2fc8fb921f4b3: Neither blocks nor log segments found. Creating new log.
I20260923 15:01:53.977455 18792 log.cc:824] T 00000000000000000000000000000000 P 5547f1c032d348338ec2fc8fb921f4b3: Log is configured to *not* fsync() on all Append() calls
I20260923 15:01:53.978374 18792 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 5547f1c032d348338ec2fc8fb921f4b3: No bootstrap required, opened a new log
I20260923 15:01:53.978394 13614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKC4soy/build/release/bin/kudu as pid 18725
I20260923 15:01:53.978528 13614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790175691491353-13614-0/minicluster-data/master-0/wal/instance
I20260923 15:01:53.978732 13614 external_mini_cluster.cc:1428] Setting key 5f3d3424e438c55ed3dda5e31c494b72
I20260923 15:01:53.980566 18792 raft_consensus.cc:348] T 00000000000000000000000000000000 P 5547f1c032d348338ec2fc8fb921f4b3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5547f1c032d348338ec2fc8fb921f4b3" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 43211 } }
I20260923 15:01:53.980711 18792 raft_consensus.cc:374] T 00000000000000000000000000000000 P 5547f1c032d348338ec2fc8fb921f4b3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:01:53.980733 18792 raft_consensus.cc:729] T 00000000000000000000000000000000 P 5547f1c032d348338ec2fc8fb921f4b3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5547f1c032d348338ec2fc8fb921f4b3, State: Initialized, Role: FOLLOWER
I20260923 15:01:53.980870 18792 consensus_queue.cc:260] T 00000000000000000000000000000000 P 5547f1c032d348338ec2fc8fb921f4b3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5547f1c032d348338ec2fc8fb921f4b3" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 43211 } }
I20260923 15:01:53.980957 18792 raft_consensus.cc:388] T 00000000000000000000000000000000 P 5547f1c032d348338ec2fc8fb921f4b3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 15:01:53.980993 18792 raft_consensus.cc:482] T 00000000000000000000000000000000 P 5547f1c032d348338ec2fc8fb921f4b3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 15:01:53.981042 18792 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 5547f1c032d348338ec2fc8fb921f4b3 [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:01:53.981624 18792 raft_consensus.cc:504] T 00000000000000000000000000000000 P 5547f1c032d348338ec2fc8fb921f4b3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5547f1c032d348338ec2fc8fb921f4b3" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 43211 } }
I20260923 15:01:53.981710 18792 leader_election.cc:302] T 00000000000000000000000000000000 P 5547f1c032d348338ec2fc8fb921f4b3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 5547f1c032d348338ec2fc8fb921f4b3; no voters:
I20260923 15:01:53.981877 18792 leader_election.cc:288] T 00000000000000000000000000000000 P 5547f1c032d348338ec2fc8fb921f4b3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 15:01:53.981964 18797 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 5547f1c032d348338ec2fc8fb921f4b3 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 15:01:53.982142 18797 raft_consensus.cc:686] T 00000000000000000000000000000000 P 5547f1c032d348338ec2fc8fb921f4b3 [term 1 LEADER]: Becoming Leader. State: Replica: 5547f1c032d348338ec2fc8fb921f4b3, State: Running, Role: LEADER
I20260923 15:01:53.982185 18792 sys_catalog.cc:564] T 00000000000000000000000000000000 P 5547f1c032d348338ec2fc8fb921f4b3 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 15:01:53.982249 18797 consensus_queue.cc:237] T 00000000000000000000000000000000 P 5547f1c032d348338ec2fc8fb921f4b3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5547f1c032d348338ec2fc8fb921f4b3" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 43211 } }
I20260923 15:01:53.982641 18799 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5547f1c032d348338ec2fc8fb921f4b3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 5547f1c032d348338ec2fc8fb921f4b3. Latest consensus state: current_term: 1 leader_uuid: "5547f1c032d348338ec2fc8fb921f4b3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5547f1c032d348338ec2fc8fb921f4b3" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 43211 } } }
I20260923 15:01:53.982641 18798 sys_catalog.cc:455] T 00000000000000000000000000000000 P 5547f1c032d348338ec2fc8fb921f4b3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "5547f1c032d348338ec2fc8fb921f4b3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "5547f1c032d348338ec2fc8fb921f4b3" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 43211 } } }
I20260923 15:01:53.982873 18798 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5547f1c032d348338ec2fc8fb921f4b3 [sys.catalog]: This master's current role is: LEADER
I20260923 15:01:53.983215 18799 sys_catalog.cc:458] T 00000000000000000000000000000000 P 5547f1c032d348338ec2fc8fb921f4b3 [sys.catalog]: This master's current role is: LEADER
I20260923 15:01:53.983423 18806 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 15:01:53.984292 18806 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 15:01:53.986366 18806 catalog_manager.cc:1347] Generated new cluster ID: 2c9526b6c9124baa80e152833149024d
I20260923 15:01:53.986419 18806 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 15:01:53.987031 18796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:53.979405 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44856 (local address 127.13.75.190:43211)
0923 15:01:53.979668 (+ 263us) server_negotiation.cc:206] Beginning negotiation
0923 15:01:53.979677 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0923 15:01:53.979711 (+ 34us) server_negotiation.cc:407] Connection header received
0923 15:01:53.980562 (+ 851us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:01:53.980571 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:01:53.980650 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:01:53.980970 (+ 320us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:01:53.981318 (+ 348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:53.983313 (+ 1995us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:53.984673 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:53.985211 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:53.985385 (+ 174us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:01:53.986159 (+ 774us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:01:53.986163 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:01:53.986164 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 15:01:53.986397 (+ 233us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:01:53.986449 (+ 52us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 15:01:53.986466 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:01:53.986556 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0923 15:01:53.986659 (+ 103us) server_negotiation.cc:299] Negotiation successful
0923 15:01:53.986751 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":172,"thread_start_us":85,"threads_started":1}
I20260923 15:01:53.988159 13614 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 15:01:53.993723 18806 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 15:01:53.994320 18806 catalog_manager.cc:1504] Loading token signing keys...
I20260923 15:01:54.000439 18806 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 5547f1c032d348338ec2fc8fb921f4b3: Generated new TSK 0
I20260923 15:01:54.000761 18806 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260923 15:01:54.025875 18822 webserver.cc:533] Webserver: sslize failed: error:14094415:SSL routines:ssl3_read_bytes:sslv3 alert certificate expired
E20260923 15:01:54.026015 18822 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260923 15:01:54.026248 18796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:54.010202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44860 (local address 127.13.75.190:43211)
0923 15:01:54.010256 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0923 15:01:54.010258 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:01:54.010389 (+ 131us) server_negotiation.cc:407] Connection header received
0923 15:01:54.010542 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:01:54.010543 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:01:54.010553 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:01:54.010613 (+ 60us) server_negotiation.cc:226] Negotiated authn=JWT
0923 15:01:54.010839 (+ 226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:54.011406 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:54.012691 (+ 1285us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:54.012904 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:54.012968 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:01:54.013299 (+ 331us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0923 15:01:54.026004 (+ 12705us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34773/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0923 15:01:54.026179 (+ 175us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:44860: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34773/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"server-negotiator.queue_time_us":23}
W20260923 15:01:54.026309 18796 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:44860: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34773/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
W20260923 15:01:54.026641 18821 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 15:01:54.010076 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:43211 (local address 127.0.0.1:44860)
0923 15:01:54.010299 (+ 223us) negotiation.cc:107] Waiting for socket to connect
0923 15:01:54.010313 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0923 15:01:54.010401 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:01:54.010659 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:01:54.010660 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:01:54.010662 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0923 15:01:54.010741 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:01:54.010744 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:01:54.011517 (+ 773us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:01:54.011519 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:01:54.012571 (+ 1052us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:01:54.012575 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:01:54.013024 (+ 449us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:01:54.013026 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:01:54.013104 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:01:54.013134 (+ 30us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0923 15:01:54.026237 (+ 13103us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34773/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
0923 15:01:54.026356 (+ 119us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.13.75.190:43211: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34773/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":102,"threads_started":1}
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:597: Failure
Failed
Bad status: Runtime error: Could not connect to the cluster: Client connection negotiation failed: client connection to 127.13.75.190:43211: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:34773/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: certificate has expired
Google Test trace:
/data0/somelongdirectorytoavoidrpathissues/src/kudu/src/kudu/integration-tests/security-itest.cc:593: Valid JWT
I20260923 15:01:54.027088 13614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKC4soy/build/release/bin/kudu with pid 18725
2026-09-23T15:01:54Z chronyd exiting
I20260923 15:01:54.063625 13614 test_util.cc:175] -----------------------------------------------
I20260923 15:01:54.063745 13614 test_util.cc:176] Had fatal failures, leaving test files at /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniCluster.1790175691491353-13614-0
[ FAILED ] SecurityITest.TestJwtMiniCluster (231 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithInvalidCert
I20260923 15:01:54.064612 13614 mini_oidc.cc:152] Starting JWKS server
I20260923 15:01:54.065388 13614 webserver.cc:460] Webserver started at https://127.0.0.1:33475/ using document root <none> and password file <none>
I20260923 15:01:54.065481 13614 mini_oidc.cc:189] Starting OIDC Discovery server
I20260923 15:01:54.065694 13614 webserver.cc:460] Webserver started at http://127.0.0.1:34307/ using document root <none> and password file <none>
2026-09-23T15:01:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T15:01:54Z Disabled control of system clock
I20260923 15:01:54.074272 13614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKC4soy/build/release/bin/kudu
/tmp/dist-test-taskKC4soy/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790175691491353-13614-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790175691491353-13614-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790175691491353-13614-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790175691491353-13614-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.75.190:46065
--webserver_interface=127.13.75.190
--webserver_port=0
--builtin_ntp_servers=127.13.75.148:44577
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.75.190:46065
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33475/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790175691491353-13614-0/testchainca.cert with env {}
W20260923 15:01:54.158001 18828 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:01:54.158237 18828 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:01:54.158262 18828 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:01:54.159569 18828 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 15:01:54.159618 18828 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:01:54.159632 18828 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:01:54.159646 18828 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 15:01:54.159657 18828 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260923 15:01:54.159670 18828 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260923 15:01:54.159682 18828 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:33475/jwks.json
I20260923 15:01:54.161378 18828 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.75.148:44577
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790175691491353-13614-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790175691491353-13614-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.75.190:46065
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.75.190:46065
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:33475/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790175691491353-13614-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.75.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790175691491353-13614-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18828
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790175691491353-13614-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:01:54.161691 18828 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:01:54.161954 18828 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:01:54.167243 18836 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:01:54.167387 18828 server_base.cc:1034] running on GCE node
W20260923 15:01:54.167235 18833 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:01:54.167265 18834 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:01:54.167783 18828 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:01:54.168093 18828 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:01:54.169263 18828 hybrid_clock.cc:648] HybridClock initialized: now 1790175714169238 us; error 40 us; skew 500 ppm
I20260923 15:01:54.170742 18828 webserver.cc:460] Webserver started at http://127.13.75.190:33967/ using document root <none> and password file <none>
I20260923 15:01:54.171099 18828 fs_manager.cc:359] Metadata directory not provided
I20260923 15:01:54.171157 18828 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:01:54.171253 18828 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:01:54.172593 18828 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790175691491353-13614-0/minicluster-data/master-0/data/instance:
uuid: "4602f723de75432e940255298d673447"
format_stamp: "Formatted at 2026-09-23 15:01:54 on dist-test-slave-cnb0"
server_key: "2dad11c41a4c1b26075733cb1312d265"
server_key_iv: "a92dc0ca1ca31ffa59349f49ac6f1043"
server_key_version: "encryptionkey@0"
I20260923 15:01:54.172952 18828 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790175691491353-13614-0/minicluster-data/master-0/wal/instance:
uuid: "4602f723de75432e940255298d673447"
format_stamp: "Formatted at 2026-09-23 15:01:54 on dist-test-slave-cnb0"
server_key: "2dad11c41a4c1b26075733cb1312d265"
server_key_iv: "a92dc0ca1ca31ffa59349f49ac6f1043"
server_key_version: "encryptionkey@0"
I20260923 15:01:54.174382 18828 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 15:01:54.175377 18842 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:01:54.175662 18828 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260923 15:01:54.175768 18828 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790175691491353-13614-0/minicluster-data/master-0/data,/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790175691491353-13614-0/minicluster-data/master-0/wal
uuid: "4602f723de75432e940255298d673447"
format_stamp: "Formatted at 2026-09-23 15:01:54 on dist-test-slave-cnb0"
server_key: "2dad11c41a4c1b26075733cb1312d265"
server_key_iv: "a92dc0ca1ca31ffa59349f49ac6f1043"
server_key_version: "encryptionkey@0"
I20260923 15:01:54.175849 18828 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790175691491353-13614-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790175691491353-13614-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790175691491353-13614-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:01:54.193045 18828 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:01:54.193486 18828 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:01:54.193655 18828 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:01:54.198434 18828 rpc_server.cc:307] RPC server started. Bound to: 127.13.75.190:46065
I20260923 15:01:54.198493 18894 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.75.190:46065 every 8 connection(s)
I20260923 15:01:54.198953 18828 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790175691491353-13614-0/minicluster-data/master-0/data/info.pb
I20260923 15:01:54.199229 13614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKC4soy/build/release/bin/kudu as pid 18828
I20260923 15:01:54.199347 13614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithInvalidCert.1790175691491353-13614-0/minicluster-data/master-0/wal/instance
I20260923 15:01:54.199543 13614 external_mini_cluster.cc:1428] Setting key 07873bee3066310c2d7d19e13938f84f
I20260923 15:01:54.200060 18895 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:01:54.203610 18895 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4602f723de75432e940255298d673447: Bootstrap starting.
I20260923 15:01:54.204332 18895 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 4602f723de75432e940255298d673447: Neither blocks nor log segments found. Creating new log.
I20260923 15:01:54.204651 18895 log.cc:824] T 00000000000000000000000000000000 P 4602f723de75432e940255298d673447: Log is configured to *not* fsync() on all Append() calls
I20260923 15:01:54.205498 18895 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 4602f723de75432e940255298d673447: No bootstrap required, opened a new log
I20260923 15:01:54.206624 18898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:54.200221 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57936 (local address 127.13.75.190:46065)
0923 15:01:54.200889 (+ 668us) server_negotiation.cc:206] Beginning negotiation
0923 15:01:54.200898 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0923 15:01:54.200926 (+ 28us) server_negotiation.cc:407] Connection header received
0923 15:01:54.201664 (+ 738us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:01:54.201673 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:01:54.201747 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:01:54.201889 (+ 142us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:01:54.202190 (+ 301us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:54.203312 (+ 1122us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:54.204432 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:54.204966 (+ 534us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:54.205084 (+ 118us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:01:54.205877 (+ 793us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:01:54.205881 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:01:54.205881 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 15:01:54.206050 (+ 169us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:01:54.206084 (+ 34us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 15:01:54.206095 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:01:54.206149 (+ 54us) server_negotiation.cc:1035] Waiting for connection context
0923 15:01:54.206258 (+ 109us) server_negotiation.cc:299] Negotiation successful
0923 15:01:54.206352 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":573,"thread_start_us":538,"threads_started":1}
I20260923 15:01:54.207943 13614 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 15:01:54.208189 18895 raft_consensus.cc:348] T 00000000000000000000000000000000 P 4602f723de75432e940255298d673447 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4602f723de75432e940255298d673447" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 46065 } }
I20260923 15:01:54.208361 18895 raft_consensus.cc:374] T 00000000000000000000000000000000 P 4602f723de75432e940255298d673447 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:01:54.208410 18895 raft_consensus.cc:729] T 00000000000000000000000000000000 P 4602f723de75432e940255298d673447 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4602f723de75432e940255298d673447, State: Initialized, Role: FOLLOWER
I20260923 15:01:54.208534 18895 consensus_queue.cc:260] T 00000000000000000000000000000000 P 4602f723de75432e940255298d673447 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4602f723de75432e940255298d673447" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 46065 } }
I20260923 15:01:54.208608 18895 raft_consensus.cc:388] T 00000000000000000000000000000000 P 4602f723de75432e940255298d673447 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 15:01:54.208643 18895 raft_consensus.cc:482] T 00000000000000000000000000000000 P 4602f723de75432e940255298d673447 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 15:01:54.208695 18895 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 4602f723de75432e940255298d673447 [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:01:54.209465 18895 raft_consensus.cc:504] T 00000000000000000000000000000000 P 4602f723de75432e940255298d673447 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4602f723de75432e940255298d673447" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 46065 } }
I20260923 15:01:54.209592 18895 leader_election.cc:302] T 00000000000000000000000000000000 P 4602f723de75432e940255298d673447 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 4602f723de75432e940255298d673447; no voters:
I20260923 15:01:54.209769 18895 leader_election.cc:288] T 00000000000000000000000000000000 P 4602f723de75432e940255298d673447 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 15:01:54.209869 18900 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 4602f723de75432e940255298d673447 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 15:01:54.210078 18895 sys_catalog.cc:564] T 00000000000000000000000000000000 P 4602f723de75432e940255298d673447 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 15:01:54.210187 18900 raft_consensus.cc:686] T 00000000000000000000000000000000 P 4602f723de75432e940255298d673447 [term 1 LEADER]: Becoming Leader. State: Replica: 4602f723de75432e940255298d673447, State: Running, Role: LEADER
I20260923 15:01:54.210307 18900 consensus_queue.cc:237] T 00000000000000000000000000000000 P 4602f723de75432e940255298d673447 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4602f723de75432e940255298d673447" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 46065 } }
I20260923 15:01:54.210750 18901 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4602f723de75432e940255298d673447 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "4602f723de75432e940255298d673447" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4602f723de75432e940255298d673447" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 46065 } } }
I20260923 15:01:54.210844 18901 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4602f723de75432e940255298d673447 [sys.catalog]: This master's current role is: LEADER
I20260923 15:01:54.211048 18901 sys_catalog.cc:455] T 00000000000000000000000000000000 P 4602f723de75432e940255298d673447 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 4602f723de75432e940255298d673447. Latest consensus state: current_term: 1 leader_uuid: "4602f723de75432e940255298d673447" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "4602f723de75432e940255298d673447" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 46065 } } }
I20260923 15:01:54.211120 18909 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 15:01:54.211146 18901 sys_catalog.cc:458] T 00000000000000000000000000000000 P 4602f723de75432e940255298d673447 [sys.catalog]: This master's current role is: LEADER
I20260923 15:01:54.211822 18909 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 15:01:54.213624 18909 catalog_manager.cc:1347] Generated new cluster ID: 294fe16bedd84f36802c8fad90c0b745
I20260923 15:01:54.213719 18909 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 15:01:54.225095 18909 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 15:01:54.225819 18909 catalog_manager.cc:1504] Loading token signing keys...
I20260923 15:01:54.229919 18909 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 4602f723de75432e940255298d673447: Generated new TSK 0
I20260923 15:01:54.230140 18909 catalog_manager.cc:1514] Initializing in-progress tserver states...
E20260923 15:01:54.252210 18925 webserver.cc:533] Webserver: sslize failed: error:14094418:SSL routines:ssl3_read_bytes:tlsv1 alert unknown ca
E20260923 15:01:54.252308 18925 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
I20260923 15:01:54.252638 18898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:54.238025 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57938 (local address 127.13.75.190:46065)
0923 15:01:54.238069 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0923 15:01:54.238073 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 15:01:54.238121 (+ 48us) server_negotiation.cc:407] Connection header received
0923 15:01:54.238217 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:01:54.238218 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:01:54.238229 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:01:54.238292 (+ 63us) server_negotiation.cc:226] Negotiated authn=JWT
0923 15:01:54.238448 (+ 156us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:54.239050 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:54.240175 (+ 1125us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:54.240369 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:54.240426 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:01:54.240623 (+ 197us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0923 15:01:54.252443 (+ 11820us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33475/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0923 15:01:54.252584 (+ 141us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:57938: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33475/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"server-negotiator.queue_time_us":21}
W20260923 15:01:54.252722 18898 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:57938: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33475/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
W20260923 15:01:54.252863 18924 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 15:01:54.237876 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:46065 (local address 127.0.0.1:57938)
0923 15:01:54.238057 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0923 15:01:54.238068 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 15:01:54.238143 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:01:54.238312 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:01:54.238312 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:01:54.238315 (+ 3us) client_negotiation.cc:189] Negotiated authn=JWT
0923 15:01:54.238376 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:01:54.238379 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:01:54.239128 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:01:54.239130 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:01:54.240081 (+ 951us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:01:54.240085 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:01:54.240453 (+ 368us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:01:54.240454 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:01:54.240517 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:01:54.240523 (+ 6us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0923 15:01:54.252580 (+ 12057us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33475/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
0923 15:01:54.252733 (+ 153us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.13.75.190:46065: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:33475/jwks.json': curl error: SSL peer certificate or SSH remote key was not OK: SSL certificate problem: unable to get local issuer certificate
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":104,"threads_started":1}
I20260923 15:01:54.253064 13614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKC4soy/build/release/bin/kudu with pid 18828
2026-09-23T15:01:54Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithInvalidCert (227 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithUntrustedCert
I20260923 15:01:54.292143 13614 mini_oidc.cc:152] Starting JWKS server
I20260923 15:01:54.292838 13614 webserver.cc:460] Webserver started at https://127.0.0.1:42049/ using document root <none> and password file <none>
I20260923 15:01:54.292886 13614 mini_oidc.cc:189] Starting OIDC Discovery server
I20260923 15:01:54.293066 13614 webserver.cc:460] Webserver started at http://127.0.0.1:41767/ using document root <none> and password file <none>
2026-09-23T15:01:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T15:01:54Z Disabled control of system clock
I20260923 15:01:54.302191 13614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKC4soy/build/release/bin/kudu
/tmp/dist-test-taskKC4soy/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790175691491353-13614-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790175691491353-13614-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790175691491353-13614-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790175691491353-13614-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.75.190:40035
--webserver_interface=127.13.75.190
--webserver_port=0
--builtin_ntp_servers=127.13.75.148:33317
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.75.190:40035
--enable_jwt_token_auth=true
--jwks_url=https://localhost:42049/jwks.json
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260923 15:01:54.396598 18931 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:01:54.396873 18931 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:01:54.396914 18931 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:01:54.398243 18931 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 15:01:54.398327 18931 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:01:54.398356 18931 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:01:54.398399 18931 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 15:01:54.398427 18931 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260923 15:01:54.398454 18931 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260923 15:01:54.398476 18931 flags.cc:425] Enabled experimental flag: --jwks_url=https://localhost:42049/jwks.json
I20260923 15:01:54.400115 18931 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.75.148:33317
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790175691491353-13614-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790175691491353-13614-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.75.190:40035
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.75.190:40035
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=https://localhost:42049/jwks.json
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790175691491353-13614-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.75.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.18931
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790175691491353-13614-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:01:54.400440 18931 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:01:54.400769 18931 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:01:54.406327 18939 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:01:54.406386 18936 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:01:54.406352 18937 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:01:54.406490 18931 server_base.cc:1034] running on GCE node
I20260923 15:01:54.406750 18931 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:01:54.407271 18931 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:01:54.408474 18931 hybrid_clock.cc:648] HybridClock initialized: now 1790175714408450 us; error 46 us; skew 500 ppm
I20260923 15:01:54.409745 18931 webserver.cc:460] Webserver started at http://127.13.75.190:43619/ using document root <none> and password file <none>
I20260923 15:01:54.410014 18931 fs_manager.cc:359] Metadata directory not provided
I20260923 15:01:54.410073 18931 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:01:54.410256 18931 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:01:54.411448 18931 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790175691491353-13614-0/minicluster-data/master-0/data/instance:
uuid: "456e5bd8a3644bef8a1ed811dc47c54f"
format_stamp: "Formatted at 2026-09-23 15:01:54 on dist-test-slave-cnb0"
server_key: "ae740efd64b093434d4d45ba40f5ad03"
server_key_iv: "9aebe537f9b2e5041f3d21b80282eaa8"
server_key_version: "encryptionkey@0"
I20260923 15:01:54.411813 18931 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790175691491353-13614-0/minicluster-data/master-0/wal/instance:
uuid: "456e5bd8a3644bef8a1ed811dc47c54f"
format_stamp: "Formatted at 2026-09-23 15:01:54 on dist-test-slave-cnb0"
server_key: "ae740efd64b093434d4d45ba40f5ad03"
server_key_iv: "9aebe537f9b2e5041f3d21b80282eaa8"
server_key_version: "encryptionkey@0"
I20260923 15:01:54.413201 18931 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 15:01:54.414165 18945 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:01:54.414394 18931 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:01:54.414566 18931 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790175691491353-13614-0/minicluster-data/master-0/data,/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790175691491353-13614-0/minicluster-data/master-0/wal
uuid: "456e5bd8a3644bef8a1ed811dc47c54f"
format_stamp: "Formatted at 2026-09-23 15:01:54 on dist-test-slave-cnb0"
server_key: "ae740efd64b093434d4d45ba40f5ad03"
server_key_iv: "9aebe537f9b2e5041f3d21b80282eaa8"
server_key_version: "encryptionkey@0"
I20260923 15:01:54.414662 18931 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790175691491353-13614-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790175691491353-13614-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790175691491353-13614-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:01:54.434448 18931 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:01:54.434824 18931 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:01:54.435019 18931 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:01:54.438807 18931 rpc_server.cc:307] RPC server started. Bound to: 127.13.75.190:40035
I20260923 15:01:54.438908 18997 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.75.190:40035 every 8 connection(s)
I20260923 15:01:54.439272 18931 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790175691491353-13614-0/minicluster-data/master-0/data/info.pb
I20260923 15:01:54.439985 18998 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:01:54.442845 18998 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 456e5bd8a3644bef8a1ed811dc47c54f: Bootstrap starting.
I20260923 15:01:54.443488 18998 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 456e5bd8a3644bef8a1ed811dc47c54f: Neither blocks nor log segments found. Creating new log.
I20260923 15:01:54.443800 18998 log.cc:824] T 00000000000000000000000000000000 P 456e5bd8a3644bef8a1ed811dc47c54f: Log is configured to *not* fsync() on all Append() calls
I20260923 15:01:54.444633 18998 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 456e5bd8a3644bef8a1ed811dc47c54f: No bootstrap required, opened a new log
I20260923 15:01:54.446249 18998 raft_consensus.cc:348] T 00000000000000000000000000000000 P 456e5bd8a3644bef8a1ed811dc47c54f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "456e5bd8a3644bef8a1ed811dc47c54f" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 40035 } }
I20260923 15:01:54.446419 18998 raft_consensus.cc:374] T 00000000000000000000000000000000 P 456e5bd8a3644bef8a1ed811dc47c54f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:01:54.446472 18998 raft_consensus.cc:729] T 00000000000000000000000000000000 P 456e5bd8a3644bef8a1ed811dc47c54f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 456e5bd8a3644bef8a1ed811dc47c54f, State: Initialized, Role: FOLLOWER
I20260923 15:01:54.446664 18998 consensus_queue.cc:260] T 00000000000000000000000000000000 P 456e5bd8a3644bef8a1ed811dc47c54f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "456e5bd8a3644bef8a1ed811dc47c54f" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 40035 } }
I20260923 15:01:54.446753 18998 raft_consensus.cc:388] T 00000000000000000000000000000000 P 456e5bd8a3644bef8a1ed811dc47c54f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 15:01:54.446785 18998 raft_consensus.cc:482] T 00000000000000000000000000000000 P 456e5bd8a3644bef8a1ed811dc47c54f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 15:01:54.446831 18998 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 456e5bd8a3644bef8a1ed811dc47c54f [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:01:54.447430 18998 raft_consensus.cc:504] T 00000000000000000000000000000000 P 456e5bd8a3644bef8a1ed811dc47c54f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "456e5bd8a3644bef8a1ed811dc47c54f" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 40035 } }
I20260923 15:01:54.447569 18998 leader_election.cc:302] T 00000000000000000000000000000000 P 456e5bd8a3644bef8a1ed811dc47c54f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 456e5bd8a3644bef8a1ed811dc47c54f; no voters:
I20260923 15:01:54.447784 18998 leader_election.cc:288] T 00000000000000000000000000000000 P 456e5bd8a3644bef8a1ed811dc47c54f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 15:01:54.447810 19001 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 456e5bd8a3644bef8a1ed811dc47c54f [term 1 FOLLOWER]: Leader election won for term 1
I20260923 15:01:54.447942 19001 raft_consensus.cc:686] T 00000000000000000000000000000000 P 456e5bd8a3644bef8a1ed811dc47c54f [term 1 LEADER]: Becoming Leader. State: Replica: 456e5bd8a3644bef8a1ed811dc47c54f, State: Running, Role: LEADER
I20260923 15:01:54.448033 19001 consensus_queue.cc:237] T 00000000000000000000000000000000 P 456e5bd8a3644bef8a1ed811dc47c54f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "456e5bd8a3644bef8a1ed811dc47c54f" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 40035 } }
I20260923 15:01:54.448089 18998 sys_catalog.cc:564] T 00000000000000000000000000000000 P 456e5bd8a3644bef8a1ed811dc47c54f [sys.catalog]: configured and running, proceeding with master startup.
I20260923 15:01:54.448316 13614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKC4soy/build/release/bin/kudu as pid 18931
I20260923 15:01:54.448419 13614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithUntrustedCert.1790175691491353-13614-0/minicluster-data/master-0/wal/instance
I20260923 15:01:54.448602 13614 external_mini_cluster.cc:1428] Setting key 845e24d74e9ab96967676f906adf8729
I20260923 15:01:54.449309 19003 sys_catalog.cc:455] T 00000000000000000000000000000000 P 456e5bd8a3644bef8a1ed811dc47c54f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "456e5bd8a3644bef8a1ed811dc47c54f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "456e5bd8a3644bef8a1ed811dc47c54f" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 40035 } } }
I20260923 15:01:54.449424 19003 sys_catalog.cc:458] T 00000000000000000000000000000000 P 456e5bd8a3644bef8a1ed811dc47c54f [sys.catalog]: This master's current role is: LEADER
I20260923 15:01:54.449651 19002 sys_catalog.cc:455] T 00000000000000000000000000000000 P 456e5bd8a3644bef8a1ed811dc47c54f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 456e5bd8a3644bef8a1ed811dc47c54f. Latest consensus state: current_term: 1 leader_uuid: "456e5bd8a3644bef8a1ed811dc47c54f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "456e5bd8a3644bef8a1ed811dc47c54f" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 40035 } } }
I20260923 15:01:54.449743 19002 sys_catalog.cc:458] T 00000000000000000000000000000000 P 456e5bd8a3644bef8a1ed811dc47c54f [sys.catalog]: This master's current role is: LEADER
I20260923 15:01:54.449750 19017 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 15:01:54.450277 19017 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 15:01:54.452176 19017 catalog_manager.cc:1347] Generated new cluster ID: 857e2f46b8c748eaa9afc2fb30485027
I20260923 15:01:54.452235 19017 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 15:01:54.456323 19018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:54.449607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47484 (local address 127.13.75.190:40035)
0923 15:01:54.449850 (+ 243us) server_negotiation.cc:206] Beginning negotiation
0923 15:01:54.449859 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0923 15:01:54.450079 (+ 220us) server_negotiation.cc:407] Connection header received
0923 15:01:54.450245 (+ 166us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:01:54.450255 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:01:54.450384 (+ 129us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:01:54.450563 (+ 179us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:01:54.450985 (+ 422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:54.452458 (+ 1473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:54.453626 (+ 1168us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:54.454257 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:54.454798 (+ 541us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:01:54.455308 (+ 510us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:01:54.455312 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:01:54.455313 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 15:01:54.455554 (+ 241us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:01:54.455598 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 15:01:54.455613 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:01:54.455951 (+ 338us) server_negotiation.cc:1035] Waiting for connection context
0923 15:01:54.455987 (+ 36us) server_negotiation.cc:299] Negotiation successful
0923 15:01:54.456074 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":163,"thread_start_us":67,"threads_started":1}
I20260923 15:01:54.457311 13614 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 15:01:54.461045 19017 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 15:01:54.461771 19017 catalog_manager.cc:1504] Loading token signing keys...
I20260923 15:01:54.471746 19017 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 456e5bd8a3644bef8a1ed811dc47c54f: Generated new TSK 0
I20260923 15:01:54.471987 19017 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 15:01:54.481994 19018 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:54.478118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47498 (local address 127.13.75.190:40035)
0923 15:01:54.478170 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0923 15:01:54.478173 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:01:54.478274 (+ 101us) server_negotiation.cc:407] Connection header received
0923 15:01:54.478354 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:01:54.478355 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:01:54.478366 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:01:54.478437 (+ 71us) server_negotiation.cc:226] Negotiated authn=JWT
0923 15:01:54.478577 (+ 140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:54.479142 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:54.480327 (+ 1185us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:54.480531 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:54.480578 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:01:54.480785 (+ 207us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0923 15:01:54.481780 (+ 995us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42049/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0923 15:01:54.481918 (+ 138us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:47498: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42049/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"server-negotiator.queue_time_us":30}
W20260923 15:01:54.482054 19018 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:47498: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42049/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
W20260923 15:01:54.482108 19027 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 15:01:54.477996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:40035 (local address 127.0.0.1:47498)
0923 15:01:54.478191 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0923 15:01:54.478203 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 15:01:54.478277 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:01:54.478453 (+ 176us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:01:54.478454 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:01:54.478456 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0923 15:01:54.478513 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:01:54.478516 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:01:54.479241 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:01:54.479243 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:01:54.480236 (+ 993us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:01:54.480239 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:01:54.480612 (+ 373us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:01:54.480614 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:01:54.480677 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:01:54.480684 (+ 7us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0923 15:01:54.481889 (+ 1205us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42049/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
0923 15:01:54.482002 (+ 113us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.13.75.190:40035: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'https://localhost:42049/jwks.json': curl error: Problem with the SSL CA cert (path? access rights?): error setting certificate verify locations:
CAfile: /etc/pki/tls/certs/ca-bundle.crt
CApath: none
Metrics: {"client-negotiator.queue_time_us":169,"thread_start_us":126,"threads_started":1}
I20260923 15:01:54.482328 13614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKC4soy/build/release/bin/kudu with pid 18931
E20260923 15:01:54.493754 19028 webserver.cc:533] Webserver: sslize failed:
E20260923 15:01:54.493839 19028 webserver.cc:533] Webserver: error shutting down SSL connection: error:140E0197:SSL routines:SSL_shutdown:shutdown while in init
2026-09-23T15:01:54Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithUntrustedCert (228 ms)
[ RUN ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS
2026-09-23T15:01:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T15:01:54Z Disabled control of system clock
I20260923 15:01:54.529048 13614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKC4soy/build/release/bin/kudu
/tmp/dist-test-taskKC4soy/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790175691491353-13614-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790175691491353-13614-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790175691491353-13614-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790175691491353-13614-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.75.190:38029
--webserver_interface=127.13.75.190
--webserver_port=0
--builtin_ntp_servers=127.13.75.148:35951
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.75.190:38029
--enable_jwt_token_auth=true
--jwks_url=default.url
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--trusted_certificate_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790175691491353-13614-0/testchainca.cert with env {}
W20260923 15:01:54.614850 19032 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:01:54.615109 19032 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:01:54.615135 19032 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:01:54.616478 19032 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 15:01:54.616528 19032 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:01:54.616542 19032 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:01:54.616556 19032 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 15:01:54.616598 19032 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
W20260923 15:01:54.616642 19032 flags.cc:425] Enabled experimental flag: --enable_jwt_token_auth=true
W20260923 15:01:54.616676 19032 flags.cc:425] Enabled experimental flag: --jwks_url=default.url
I20260923 15:01:54.618458 19032 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.75.148:35951
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790175691491353-13614-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790175691491353-13614-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.75.190:38029
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.75.190:38029
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--enable_jwt_token_auth=true
--jwks_url=default.url
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790175691491353-13614-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.75.190
--webserver_port=0
--trusted_certificate_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790175691491353-13614-0/testchainca.cert
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19032
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790175691491353-13614-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:01:54.618760 19032 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:01:54.619057 19032 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:01:54.625553 19040 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:01:54.625579 19037 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:01:54.625643 19038 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:01:54.625742 19032 server_base.cc:1034] running on GCE node
I20260923 15:01:54.626084 19032 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:01:54.626396 19032 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:01:54.627584 19032 hybrid_clock.cc:648] HybridClock initialized: now 1790175714627569 us; error 45 us; skew 500 ppm
I20260923 15:01:54.628858 19032 webserver.cc:460] Webserver started at http://127.13.75.190:37715/ using document root <none> and password file <none>
I20260923 15:01:54.629107 19032 fs_manager.cc:359] Metadata directory not provided
I20260923 15:01:54.629176 19032 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:01:54.629318 19032 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:01:54.630452 19032 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790175691491353-13614-0/minicluster-data/master-0/data/instance:
uuid: "1f3125148c0b4e0e8ef2306cb34184c1"
format_stamp: "Formatted at 2026-09-23 15:01:54 on dist-test-slave-cnb0"
server_key: "4ed16012811213d5061fd392346d36ce"
server_key_iv: "3b9802d4e8523292a99d236e6ee8ea95"
server_key_version: "encryptionkey@0"
I20260923 15:01:54.630834 19032 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790175691491353-13614-0/minicluster-data/master-0/wal/instance:
uuid: "1f3125148c0b4e0e8ef2306cb34184c1"
format_stamp: "Formatted at 2026-09-23 15:01:54 on dist-test-slave-cnb0"
server_key: "4ed16012811213d5061fd392346d36ce"
server_key_iv: "3b9802d4e8523292a99d236e6ee8ea95"
server_key_version: "encryptionkey@0"
I20260923 15:01:54.632205 19032 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:01:54.632975 19046 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:01:54.633152 19032 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 15:01:54.633252 19032 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790175691491353-13614-0/minicluster-data/master-0/data,/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790175691491353-13614-0/minicluster-data/master-0/wal
uuid: "1f3125148c0b4e0e8ef2306cb34184c1"
format_stamp: "Formatted at 2026-09-23 15:01:54 on dist-test-slave-cnb0"
server_key: "4ed16012811213d5061fd392346d36ce"
server_key_iv: "3b9802d4e8523292a99d236e6ee8ea95"
server_key_version: "encryptionkey@0"
I20260923 15:01:54.633322 19032 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790175691491353-13614-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790175691491353-13614-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790175691491353-13614-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:01:54.670104 19032 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:01:54.670449 19032 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:01:54.670610 19032 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:01:54.675024 19032 rpc_server.cc:307] RPC server started. Bound to: 127.13.75.190:38029
I20260923 15:01:54.675057 19098 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.75.190:38029 every 8 connection(s)
I20260923 15:01:54.675653 19032 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790175691491353-13614-0/minicluster-data/master-0/data/info.pb
I20260923 15:01:54.676690 19099 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:01:54.679157 19099 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1f3125148c0b4e0e8ef2306cb34184c1: Bootstrap starting.
I20260923 15:01:54.679738 19099 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 1f3125148c0b4e0e8ef2306cb34184c1: Neither blocks nor log segments found. Creating new log.
I20260923 15:01:54.680047 19099 log.cc:824] T 00000000000000000000000000000000 P 1f3125148c0b4e0e8ef2306cb34184c1: Log is configured to *not* fsync() on all Append() calls
I20260923 15:01:54.680883 19099 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 1f3125148c0b4e0e8ef2306cb34184c1: No bootstrap required, opened a new log
I20260923 15:01:54.682734 19099 raft_consensus.cc:348] T 00000000000000000000000000000000 P 1f3125148c0b4e0e8ef2306cb34184c1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f3125148c0b4e0e8ef2306cb34184c1" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 38029 } }
I20260923 15:01:54.682955 19099 raft_consensus.cc:374] T 00000000000000000000000000000000 P 1f3125148c0b4e0e8ef2306cb34184c1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:01:54.683030 19099 raft_consensus.cc:729] T 00000000000000000000000000000000 P 1f3125148c0b4e0e8ef2306cb34184c1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1f3125148c0b4e0e8ef2306cb34184c1, State: Initialized, Role: FOLLOWER
I20260923 15:01:54.683228 19099 consensus_queue.cc:260] T 00000000000000000000000000000000 P 1f3125148c0b4e0e8ef2306cb34184c1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f3125148c0b4e0e8ef2306cb34184c1" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 38029 } }
I20260923 15:01:54.683346 19099 raft_consensus.cc:388] T 00000000000000000000000000000000 P 1f3125148c0b4e0e8ef2306cb34184c1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 15:01:54.683395 19099 raft_consensus.cc:482] T 00000000000000000000000000000000 P 1f3125148c0b4e0e8ef2306cb34184c1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 15:01:54.683451 19099 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 1f3125148c0b4e0e8ef2306cb34184c1 [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:01:54.684028 19099 raft_consensus.cc:504] T 00000000000000000000000000000000 P 1f3125148c0b4e0e8ef2306cb34184c1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f3125148c0b4e0e8ef2306cb34184c1" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 38029 } }
I20260923 15:01:54.684156 19099 leader_election.cc:302] T 00000000000000000000000000000000 P 1f3125148c0b4e0e8ef2306cb34184c1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 1f3125148c0b4e0e8ef2306cb34184c1; no voters:
I20260923 15:01:54.684345 19099 leader_election.cc:288] T 00000000000000000000000000000000 P 1f3125148c0b4e0e8ef2306cb34184c1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 15:01:54.684388 19102 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 1f3125148c0b4e0e8ef2306cb34184c1 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 15:01:54.684516 19102 raft_consensus.cc:686] T 00000000000000000000000000000000 P 1f3125148c0b4e0e8ef2306cb34184c1 [term 1 LEADER]: Becoming Leader. State: Replica: 1f3125148c0b4e0e8ef2306cb34184c1, State: Running, Role: LEADER
I20260923 15:01:54.684614 19102 consensus_queue.cc:237] T 00000000000000000000000000000000 P 1f3125148c0b4e0e8ef2306cb34184c1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f3125148c0b4e0e8ef2306cb34184c1" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 38029 } }
I20260923 15:01:54.684626 13614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKC4soy/build/release/bin/kudu as pid 19032
I20260923 15:01:54.684770 19099 sys_catalog.cc:564] T 00000000000000000000000000000000 P 1f3125148c0b4e0e8ef2306cb34184c1 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 15:01:54.684823 13614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS.1790175691491353-13614-0/minicluster-data/master-0/wal/instance
I20260923 15:01:54.685074 13614 external_mini_cluster.cc:1428] Setting key 64fb4a38ab3839ff2c35f9b81e471ce4
I20260923 15:01:54.685057 19103 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f3125148c0b4e0e8ef2306cb34184c1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 1f3125148c0b4e0e8ef2306cb34184c1. Latest consensus state: current_term: 1 leader_uuid: "1f3125148c0b4e0e8ef2306cb34184c1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f3125148c0b4e0e8ef2306cb34184c1" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 38029 } } }
I20260923 15:01:54.685135 19103 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f3125148c0b4e0e8ef2306cb34184c1 [sys.catalog]: This master's current role is: LEADER
I20260923 15:01:54.685304 19104 sys_catalog.cc:455] T 00000000000000000000000000000000 P 1f3125148c0b4e0e8ef2306cb34184c1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "1f3125148c0b4e0e8ef2306cb34184c1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "1f3125148c0b4e0e8ef2306cb34184c1" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 38029 } } }
I20260923 15:01:54.685379 19104 sys_catalog.cc:458] T 00000000000000000000000000000000 P 1f3125148c0b4e0e8ef2306cb34184c1 [sys.catalog]: This master's current role is: LEADER
I20260923 15:01:54.685483 19106 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 15:01:54.686333 19106 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 15:01:54.689071 19106 catalog_manager.cc:1347] Generated new cluster ID: 3d4d9863dbd04cd4a143979a3a4e8bea
I20260923 15:01:54.689179 19106 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 15:01:54.691798 19113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:54.685692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33816 (local address 127.13.75.190:38029)
0923 15:01:54.686061 (+ 369us) server_negotiation.cc:206] Beginning negotiation
0923 15:01:54.686071 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0923 15:01:54.686111 (+ 40us) server_negotiation.cc:407] Connection header received
0923 15:01:54.686230 (+ 119us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:01:54.686237 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:01:54.686317 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:01:54.686790 (+ 473us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:01:54.686928 (+ 138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:54.688391 (+ 1463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:54.689529 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:54.690180 (+ 651us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:54.690637 (+ 457us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:01:54.691085 (+ 448us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:01:54.691088 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:01:54.691089 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 15:01:54.691290 (+ 201us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:01:54.691330 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 15:01:54.691344 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:01:54.691423 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0923 15:01:54.691522 (+ 99us) server_negotiation.cc:299] Negotiation successful
0923 15:01:54.691598 (+ 76us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":280,"thread_start_us":90,"threads_started":1}
I20260923 15:01:54.692766 13614 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 15:01:54.696887 19106 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 15:01:54.697391 19106 catalog_manager.cc:1504] Loading token signing keys...
I20260923 15:01:54.707288 19106 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 1f3125148c0b4e0e8ef2306cb34184c1: Generated new TSK 0
I20260923 15:01:54.707520 19106 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 15:01:54.777201 19113 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:54.712311 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33824 (local address 127.13.75.190:38029)
0923 15:01:54.712359 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0923 15:01:54.712361 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:01:54.712468 (+ 107us) server_negotiation.cc:407] Connection header received
0923 15:01:54.712527 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:01:54.712528 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:01:54.712538 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:01:54.712605 (+ 67us) server_negotiation.cc:226] Negotiated authn=JWT
0923 15:01:54.712814 (+ 209us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:54.713354 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:54.714580 (+ 1226us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:54.714764 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:54.714825 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:01:54.715045 (+ 220us) server_negotiation.cc:365] Received JWT_EXCHANGE NegotiatePB request
0923 15:01:54.776968 (+ 61923us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0923 15:01:54.777140 (+ 172us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:33824: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"server-negotiator.queue_time_us":25}
W20260923 15:01:54.777309 19113 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:33824: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
W20260923 15:01:54.777406 19128 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 15:01:54.712248 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:38029 (local address 127.0.0.1:33824)
0923 15:01:54.712388 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0923 15:01:54.712398 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 15:01:54.712460 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:01:54.712625 (+ 165us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:01:54.712626 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:01:54.712628 (+ 2us) client_negotiation.cc:189] Negotiated authn=JWT
0923 15:01:54.712694 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:01:54.712697 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:01:54.713444 (+ 747us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:01:54.713445 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:01:54.714433 (+ 988us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:01:54.714437 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:01:54.714848 (+ 411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:01:54.714849 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:01:54.714933 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:01:54.714939 (+ 6us) client_negotiation.cc:252] Sending JWT_EXCHANGE NegotiatePB request
0923 15:01:54.777131 (+ 62192us) client_negotiation.cc:283] Received error response from server: Runtime error: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
0923 15:01:54.777292 (+ 161us) negotiation.cc:327] Negotiation complete: Runtime error: Client connection negotiation failed: client connection to 127.13.75.190:38029: FATAL_INVALID_JWT: Not authorized: Failed to load JWKS: Error downloading JWKS from 'default.url': curl error: Couldn't resolve host name: Could not resolve host: default.url
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":75,"threads_started":1}
I20260923 15:01:54.777925 13614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKC4soy/build/release/bin/kudu with pid 19032
2026-09-23T15:01:54Z chronyd exiting
[ OK ] SecurityITest.TestJwtMiniClusterWithNonWorkingJWKS (274 ms)
[ RUN ] SecurityITest.TestWorldReadableKeytab
[ OK ] SecurityITest.TestWorldReadableKeytab (91 ms)
[ RUN ] SecurityITest.TestWorldReadablePrivateKey
[ OK ] SecurityITest.TestWorldReadablePrivateKey (92 ms)
[ RUN ] SecurityITest.TestCorruptKerberosCC
Loading random data
Initializing database '/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175691491353-13614-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 15:01:55 dist-test-slave-cnb0 krb5kdc[19140](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 15:01:55 dist-test-slave-cnb0 krb5kdc[19140](info): set up 2 sockets
Sep 23 15:01:55 dist-test-slave-cnb0 krb5kdc[19140](info): commencing operation
krb5kdc: starting...
W20260923 15:01:56.052446 13614 mini_kdc.cc:121] Time spent starting KDC: real 1.072s user 0.005s sys 0.000s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175691491353-13614-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175691491353-13614-0/krb5kdc/test-user.keytab.
Sep 23 15:01:56 dist-test-slave-cnb0 krb5kdc[19140](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175716, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T15:01:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T15:01:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.75.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.75.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.75.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.190.keytab.
Entry for principal kudu/127.13.75.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.190.keytab.
I20260923 15:01:56.183270 13614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKC4soy/build/release/bin/kudu
/tmp/dist-test-taskKC4soy/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175691491353-13614-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175691491353-13614-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175691491353-13614-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175691491353-13614-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.75.190:39487
--webserver_interface=127.13.75.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.190.keytab
--principal=kudu/127.13.75.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.75.148:44133
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.75.190:39487
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175691491353-13614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175691491353-13614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175691491353-13614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:01:56.265705 19153 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:01:56.265980 19153 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:01:56.266017 19153 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:01:56.267277 19153 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 15:01:56.267365 19153 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:01:56.267387 19153 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:01:56.267408 19153 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 15:01:56.267441 19153 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 15:01:56.268987 19153 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.75.148:44133
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175691491353-13614-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175691491353-13614-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.75.190:39487
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.190.keytab
--principal=kudu/127.13.75.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.75.190:39487
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175691491353-13614-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.75.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19153
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175691491353-13614-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:01:56.269285 19153 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:01:56.269531 19153 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:01:56.274657 19158 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:01:56.274622 19159 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:01:56.274622 19161 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:01:56.274804 19153 server_base.cc:1034] running on GCE node
I20260923 15:01:56.275316 19153 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:01:56.275650 19153 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:01:56.276858 19153 hybrid_clock.cc:648] HybridClock initialized: now 1790175716276824 us; error 54 us; skew 500 ppm
Sep 23 15:01:56 dist-test-slave-cnb0 krb5kdc[19140](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175716, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:01:56.280022 19153 init.cc:377] Logged in from keytab as kudu/127.13.75.190@KRBTEST.COM (short username kudu)
I20260923 15:01:56.280634 19153 webserver.cc:460] Webserver started at http://127.13.75.190:34633/ using document root <none> and password file <none>
I20260923 15:01:56.280885 19153 fs_manager.cc:359] Metadata directory not provided
I20260923 15:01:56.280961 19153 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:01:56.281126 19153 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:01:56.282219 19153 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175691491353-13614-0/minicluster-data/master-0/data/instance:
uuid: "305b5fd2962241b5b1104b24137b6efd"
format_stamp: "Formatted at 2026-09-23 15:01:56 on dist-test-slave-cnb0"
server_key: "5562e2b56ebc850f274d624d134ebad9"
server_key_iv: "b74e14911efa0ad3b854031a0b45e4bd"
server_key_version: "encryptionkey@0"
I20260923 15:01:56.282599 19153 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175691491353-13614-0/minicluster-data/master-0/wal/instance:
uuid: "305b5fd2962241b5b1104b24137b6efd"
format_stamp: "Formatted at 2026-09-23 15:01:56 on dist-test-slave-cnb0"
server_key: "5562e2b56ebc850f274d624d134ebad9"
server_key_iv: "b74e14911efa0ad3b854031a0b45e4bd"
server_key_version: "encryptionkey@0"
I20260923 15:01:56.284096 19153 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 15:01:56.284992 19168 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:01:56.285211 19153 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:01:56.285383 19153 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175691491353-13614-0/minicluster-data/master-0/data,/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175691491353-13614-0/minicluster-data/master-0/wal
uuid: "305b5fd2962241b5b1104b24137b6efd"
format_stamp: "Formatted at 2026-09-23 15:01:56 on dist-test-slave-cnb0"
server_key: "5562e2b56ebc850f274d624d134ebad9"
server_key_iv: "b74e14911efa0ad3b854031a0b45e4bd"
server_key_version: "encryptionkey@0"
I20260923 15:01:56.285492 19153 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175691491353-13614-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175691491353-13614-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175691491353-13614-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:01:56.297753 19153 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:01:56.300361 19153 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:01:56.300555 19153 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:01:56.304857 19153 rpc_server.cc:307] RPC server started. Bound to: 127.13.75.190:39487
I20260923 15:01:56.304991 19220 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.75.190:39487 every 8 connection(s)
I20260923 15:01:56.305325 19153 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175691491353-13614-0/minicluster-data/master-0/data/info.pb
I20260923 15:01:56.306299 19221 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:01:56.308638 13614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKC4soy/build/release/bin/kudu as pid 19153
I20260923 15:01:56.308769 13614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175691491353-13614-0/minicluster-data/master-0/wal/instance
I20260923 15:01:56.308880 19221 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 305b5fd2962241b5b1104b24137b6efd: Bootstrap starting.
I20260923 15:01:56.308993 13614 external_mini_cluster.cc:1428] Setting key 7f48c89f4496af250d674867396490f3
I20260923 15:01:56.309681 19221 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 305b5fd2962241b5b1104b24137b6efd: Neither blocks nor log segments found. Creating new log.
I20260923 15:01:56.310030 19221 log.cc:824] T 00000000000000000000000000000000 P 305b5fd2962241b5b1104b24137b6efd: Log is configured to *not* fsync() on all Append() calls
I20260923 15:01:56.310803 19221 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 305b5fd2962241b5b1104b24137b6efd: No bootstrap required, opened a new log
I20260923 15:01:56.312868 19221 raft_consensus.cc:348] T 00000000000000000000000000000000 P 305b5fd2962241b5b1104b24137b6efd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "305b5fd2962241b5b1104b24137b6efd" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 39487 } }
I20260923 15:01:56.313053 19221 raft_consensus.cc:374] T 00000000000000000000000000000000 P 305b5fd2962241b5b1104b24137b6efd [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:01:56.313105 19221 raft_consensus.cc:729] T 00000000000000000000000000000000 P 305b5fd2962241b5b1104b24137b6efd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 305b5fd2962241b5b1104b24137b6efd, State: Initialized, Role: FOLLOWER
I20260923 15:01:56.313274 19221 consensus_queue.cc:260] T 00000000000000000000000000000000 P 305b5fd2962241b5b1104b24137b6efd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "305b5fd2962241b5b1104b24137b6efd" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 39487 } }
I20260923 15:01:56.313364 19221 raft_consensus.cc:388] T 00000000000000000000000000000000 P 305b5fd2962241b5b1104b24137b6efd [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 15:01:56.313413 19221 raft_consensus.cc:482] T 00000000000000000000000000000000 P 305b5fd2962241b5b1104b24137b6efd [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 15:01:56.313469 19221 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 305b5fd2962241b5b1104b24137b6efd [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:01:56.314190 19221 raft_consensus.cc:504] T 00000000000000000000000000000000 P 305b5fd2962241b5b1104b24137b6efd [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "305b5fd2962241b5b1104b24137b6efd" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 39487 } }
I20260923 15:01:56.314340 19221 leader_election.cc:302] T 00000000000000000000000000000000 P 305b5fd2962241b5b1104b24137b6efd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 305b5fd2962241b5b1104b24137b6efd; no voters:
I20260923 15:01:56.314572 19221 leader_election.cc:288] T 00000000000000000000000000000000 P 305b5fd2962241b5b1104b24137b6efd [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 15:01:56.314607 19226 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 305b5fd2962241b5b1104b24137b6efd [term 1 FOLLOWER]: Leader election won for term 1
I20260923 15:01:56.314764 19226 raft_consensus.cc:686] T 00000000000000000000000000000000 P 305b5fd2962241b5b1104b24137b6efd [term 1 LEADER]: Becoming Leader. State: Replica: 305b5fd2962241b5b1104b24137b6efd, State: Running, Role: LEADER
I20260923 15:01:56.314860 19226 consensus_queue.cc:237] T 00000000000000000000000000000000 P 305b5fd2962241b5b1104b24137b6efd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "305b5fd2962241b5b1104b24137b6efd" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 39487 } }
I20260923 15:01:56.314942 19221 sys_catalog.cc:564] T 00000000000000000000000000000000 P 305b5fd2962241b5b1104b24137b6efd [sys.catalog]: configured and running, proceeding with master startup.
I20260923 15:01:56.315245 19228 sys_catalog.cc:455] T 00000000000000000000000000000000 P 305b5fd2962241b5b1104b24137b6efd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "305b5fd2962241b5b1104b24137b6efd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "305b5fd2962241b5b1104b24137b6efd" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 39487 } } }
I20260923 15:01:56.315477 19228 sys_catalog.cc:458] T 00000000000000000000000000000000 P 305b5fd2962241b5b1104b24137b6efd [sys.catalog]: This master's current role is: LEADER
I20260923 15:01:56.315408 19227 sys_catalog.cc:455] T 00000000000000000000000000000000 P 305b5fd2962241b5b1104b24137b6efd [sys.catalog]: SysCatalogTable state changed. Reason: New leader 305b5fd2962241b5b1104b24137b6efd. Latest consensus state: current_term: 1 leader_uuid: "305b5fd2962241b5b1104b24137b6efd" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "305b5fd2962241b5b1104b24137b6efd" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 39487 } } }
I20260923 15:01:56.315776 19227 sys_catalog.cc:458] T 00000000000000000000000000000000 P 305b5fd2962241b5b1104b24137b6efd [sys.catalog]: This master's current role is: LEADER
Sep 23 15:01:56 dist-test-slave-cnb0 krb5kdc[19140](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175716, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.75.190@KRBTEST.COM
I20260923 15:01:56.316159 19235 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 15:01:56.316676 19235 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 15:01:56.318528 19235 catalog_manager.cc:1347] Generated new cluster ID: ff6b8ef61db94b5785d709e57d2488b6
I20260923 15:01:56.318581 19235 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 15:01:56.320782 19224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:56.309590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42036 (local address 127.13.75.190:39487)
0923 15:01:56.309806 (+ 216us) server_negotiation.cc:206] Beginning negotiation
0923 15:01:56.309813 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0923 15:01:56.309839 (+ 26us) server_negotiation.cc:407] Connection header received
0923 15:01:56.310307 (+ 468us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:01:56.310314 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:01:56.310364 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:01:56.310529 (+ 165us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:01:56.311270 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:56.312140 (+ 870us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:56.312927 (+ 787us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:56.313490 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:56.313638 (+ 148us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:01:56.316672 (+ 3034us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:01:56.316676 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:01:56.316677 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:01:56.316689 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:01:56.318976 (+ 2287us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:01:56.319394 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:01:56.319398 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:01:56.319398 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:01:56.319517 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:01:56.319736 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:01:56.319737 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:01:56.319737 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:01:56.320091 (+ 354us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:01:56.320257 (+ 166us) server_negotiation.cc:1035] Waiting for connection context
0923 15:01:56.320377 (+ 120us) server_negotiation.cc:299] Negotiation successful
0923 15:01:56.320535 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":85,"threads_started":1}
I20260923 15:01:56.334460 19235 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 15:01:56.335395 19235 catalog_manager.cc:1504] Loading token signing keys...
I20260923 15:01:56.339913 19235 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 305b5fd2962241b5b1104b24137b6efd: Generated new TSK 0
I20260923 15:01:56.340128 19235 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.75.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.75.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.75.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.129.keytab.
Entry for principal kudu/127.13.75.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.129.keytab.
I20260923 15:01:56.371295 13614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKC4soy/build/release/bin/kudu
/tmp/dist-test-taskKC4soy/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175691491353-13614-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175691491353-13614-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175691491353-13614-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175691491353-13614-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.75.129:0
--local_ip_for_outbound_sockets=127.13.75.129
--webserver_interface=127.13.75.129
--webserver_port=0
--tserver_master_addrs=127.13.75.190:39487
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.129.keytab
--principal=kudu/127.13.75.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.75.148:44133
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175691491353-13614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175691491353-13614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175691491353-13614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:01:56.455250 19247 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:01:56.455523 19247 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:01:56.455566 19247 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:01:56.456866 19247 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:01:56.456919 19247 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:01:56.456949 19247 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.75.129
I20260923 15:01:56.458568 19247 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.75.148:44133
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175691491353-13614-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175691491353-13614-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.129.keytab
--principal=kudu/127.13.75.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.75.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175691491353-13614-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.75.129
--webserver_port=0
--tserver_master_addrs=127.13.75.190:39487
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19247
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.75.129
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175691491353-13614-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:01:56.458865 19247 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:01:56.459154 19247 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:01:56.464366 19252 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:01:56.464367 19255 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:01:56.464398 19253 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:01:56.464754 19247 server_base.cc:1034] running on GCE node
I20260923 15:01:56.464989 19247 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:01:56.465263 19247 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:01:56.466441 19247 hybrid_clock.cc:648] HybridClock initialized: now 1790175716466428 us; error 30 us; skew 500 ppm
Sep 23 15:01:56 dist-test-slave-cnb0 krb5kdc[19140](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175716, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:01:56.469308 19247 init.cc:377] Logged in from keytab as kudu/127.13.75.129@KRBTEST.COM (short username kudu)
I20260923 15:01:56.469822 19247 webserver.cc:460] Webserver started at http://127.13.75.129:35385/ using document root <none> and password file <none>
I20260923 15:01:56.470034 19247 fs_manager.cc:359] Metadata directory not provided
I20260923 15:01:56.470075 19247 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:01:56.470167 19247 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:01:56.471313 19247 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175691491353-13614-0/minicluster-data/ts-0/data/instance:
uuid: "15f2992799fc40769706e1c4a44a447f"
format_stamp: "Formatted at 2026-09-23 15:01:56 on dist-test-slave-cnb0"
server_key: "e4cce6aabf76e0225b179e73b55c22e3"
server_key_iv: "c88f17fbfad5f724987edd2f47f54327"
server_key_version: "encryptionkey@0"
I20260923 15:01:56.471639 19247 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175691491353-13614-0/minicluster-data/ts-0/wal/instance:
uuid: "15f2992799fc40769706e1c4a44a447f"
format_stamp: "Formatted at 2026-09-23 15:01:56 on dist-test-slave-cnb0"
server_key: "e4cce6aabf76e0225b179e73b55c22e3"
server_key_iv: "c88f17fbfad5f724987edd2f47f54327"
server_key_version: "encryptionkey@0"
I20260923 15:01:56.473032 19247 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 15:01:56.473903 19262 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:01:56.474141 19247 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:01:56.474232 19247 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175691491353-13614-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175691491353-13614-0/minicluster-data/ts-0/wal
uuid: "15f2992799fc40769706e1c4a44a447f"
format_stamp: "Formatted at 2026-09-23 15:01:56 on dist-test-slave-cnb0"
server_key: "e4cce6aabf76e0225b179e73b55c22e3"
server_key_iv: "c88f17fbfad5f724987edd2f47f54327"
server_key_version: "encryptionkey@0"
I20260923 15:01:56.474301 19247 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175691491353-13614-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175691491353-13614-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175691491353-13614-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:01:56.494426 19247 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:01:56.496904 19247 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:01:56.497033 19247 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:01:56.497228 19247 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 15:01:56.497843 19247 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 15:01:56.497889 19247 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 15:01:56.497915 19247 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 15:01:56.497927 19247 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 15:01:56.504067 19247 rpc_server.cc:307] RPC server started. Bound to: 127.13.75.129:44681
I20260923 15:01:56.504170 19375 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.75.129:44681 every 8 connection(s)
I20260923 15:01:56.504506 19247 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175691491353-13614-0/minicluster-data/ts-0/data/info.pb
I20260923 15:01:56.506502 13614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKC4soy/build/release/bin/kudu as pid 19247
I20260923 15:01:56.506662 13614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175691491353-13614-0/minicluster-data/ts-0/wal/instance
I20260923 15:01:56.506856 13614 external_mini_cluster.cc:1428] Setting key cee6cc80955cca08713db4599f7608c9
Sep 23 15:01:56 dist-test-slave-cnb0 krb5kdc[19140](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175716, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.129@KRBTEST.COM for kudu/127.13.75.190@KRBTEST.COM
I20260923 15:01:56.516245 19224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:56.505313 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.129:37149 (local address 127.13.75.190:39487)
0923 15:01:56.505386 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0923 15:01:56.505388 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:01:56.505586 (+ 198us) server_negotiation.cc:407] Connection header received
0923 15:01:56.506592 (+ 1006us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:01:56.506593 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:01:56.506604 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:01:56.506709 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:01:56.507549 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:56.508045 (+ 496us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:56.509081 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:56.509325 (+ 244us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:56.509412 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:01:56.512579 (+ 3167us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:01:56.512582 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:01:56.512582 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:01:56.512592 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:01:56.514350 (+ 1758us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:01:56.514933 (+ 583us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:01:56.514935 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:01:56.514935 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:01:56.515030 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:01:56.515352 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:01:56.515354 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:01:56.515354 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:01:56.515519 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:01:56.515608 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0923 15:01:56.516031 (+ 423us) server_negotiation.cc:299] Negotiation successful
0923 15:01:56.516132 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260923 15:01:56.516510 19378 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:56.505267 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:39487 (local address 127.13.75.129:37149)
0923 15:01:56.505441 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0923 15:01:56.505485 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0923 15:01:56.506480 (+ 995us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:01:56.506839 (+ 359us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:01:56.506843 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:01:56.507197 (+ 354us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:01:56.507393 (+ 196us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:01:56.507398 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:01:56.508163 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:01:56.508164 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:01:56.508978 (+ 814us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:01:56.508980 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:01:56.509439 (+ 459us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:01:56.509440 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:01:56.509587 (+ 147us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:01:56.510309 (+ 722us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:01:56.510321 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:01:56.512463 (+ 2142us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:01:56.514476 (+ 2013us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:01:56.514483 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:01:56.514485 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:01:56.514728 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:01:56.515135 (+ 407us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:01:56.515135 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:01:56.515136 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:01:56.515251 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:01:56.515625 (+ 374us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:01:56.515633 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:01:56.515858 (+ 225us) client_negotiation.cc:769] Sending connection context
0923 15:01:56.515979 (+ 121us) client_negotiation.cc:240] Negotiation successful
0923 15:01:56.516236 (+ 257us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":91,"threads_started":1}
WARNING: no policy specified for kudu/127.13.75.130@KRBTEST.COM; defaulting to no policy
I20260923 15:01:56.517140 19376 heartbeater.cc:347] Connected to a master server at 127.13.75.190:39487
I20260923 15:01:56.517242 19376 heartbeater.cc:464] Registering TS with master...
I20260923 15:01:56.517472 19376 heartbeater.cc:511] Master 127.13.75.190:39487 requested a full tablet report, sending...
I20260923 15:01:56.518091 19185 ts_manager.cc:194] Registered new tserver with Master: 15f2992799fc40769706e1c4a44a447f (127.13.75.129:44681)
I20260923 15:01:56.519119 19185 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.75.129@KRBTEST.COM'} at 127.13.75.129:37149
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.75.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.75.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.130.keytab.
Entry for principal kudu/127.13.75.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.130.keytab.
I20260923 15:01:56.538719 13614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKC4soy/build/release/bin/kudu
/tmp/dist-test-taskKC4soy/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175691491353-13614-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175691491353-13614-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175691491353-13614-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175691491353-13614-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.75.130:0
--local_ip_for_outbound_sockets=127.13.75.130
--webserver_interface=127.13.75.130
--webserver_port=0
--tserver_master_addrs=127.13.75.190:39487
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.130.keytab
--principal=kudu/127.13.75.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.75.148:44133
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175691491353-13614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175691491353-13614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175691491353-13614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:01:56.626797 19381 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:01:56.627038 19381 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:01:56.627063 19381 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:01:56.628223 19381 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:01:56.628265 19381 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:01:56.628293 19381 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.75.130
I20260923 15:01:56.629753 19381 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.75.148:44133
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175691491353-13614-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175691491353-13614-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.130.keytab
--principal=kudu/127.13.75.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.75.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175691491353-13614-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.75.130
--webserver_port=0
--tserver_master_addrs=127.13.75.190:39487
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19381
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.75.130
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175691491353-13614-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:01:56.629961 19381 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:01:56.630147 19381 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:01:56.635882 19387 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:01:56.635887 19389 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:01:56.635916 19386 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:01:56.636155 19381 server_base.cc:1034] running on GCE node
I20260923 15:01:56.636494 19381 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:01:56.636767 19381 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:01:56.637943 19381 hybrid_clock.cc:648] HybridClock initialized: now 1790175716637926 us; error 57 us; skew 500 ppm
Sep 23 15:01:56 dist-test-slave-cnb0 krb5kdc[19140](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175716, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:01:56.640765 19381 init.cc:377] Logged in from keytab as kudu/127.13.75.130@KRBTEST.COM (short username kudu)
I20260923 15:01:56.641321 19381 webserver.cc:460] Webserver started at http://127.13.75.130:38889/ using document root <none> and password file <none>
I20260923 15:01:56.641584 19381 fs_manager.cc:359] Metadata directory not provided
I20260923 15:01:56.641659 19381 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:01:56.641815 19381 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:01:56.642966 19381 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175691491353-13614-0/minicluster-data/ts-1/data/instance:
uuid: "c329a2dfbf8242848b8e1a05139af590"
format_stamp: "Formatted at 2026-09-23 15:01:56 on dist-test-slave-cnb0"
server_key: "3a02d1f08e4c3bad56113431b712f804"
server_key_iv: "7bcfb11c5ce14487f165322ed5113bbf"
server_key_version: "encryptionkey@0"
I20260923 15:01:56.643361 19381 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175691491353-13614-0/minicluster-data/ts-1/wal/instance:
uuid: "c329a2dfbf8242848b8e1a05139af590"
format_stamp: "Formatted at 2026-09-23 15:01:56 on dist-test-slave-cnb0"
server_key: "3a02d1f08e4c3bad56113431b712f804"
server_key_iv: "7bcfb11c5ce14487f165322ed5113bbf"
server_key_version: "encryptionkey@0"
I20260923 15:01:56.644817 19381 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 15:01:56.645876 19396 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:01:56.646126 19381 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:01:56.646256 19381 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175691491353-13614-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175691491353-13614-0/minicluster-data/ts-1/wal
uuid: "c329a2dfbf8242848b8e1a05139af590"
format_stamp: "Formatted at 2026-09-23 15:01:56 on dist-test-slave-cnb0"
server_key: "3a02d1f08e4c3bad56113431b712f804"
server_key_iv: "7bcfb11c5ce14487f165322ed5113bbf"
server_key_version: "encryptionkey@0"
I20260923 15:01:56.646350 19381 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175691491353-13614-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175691491353-13614-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175691491353-13614-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:01:56.664549 19381 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:01:56.667232 19381 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:01:56.667436 19381 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:01:56.667724 19381 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 15:01:56.668133 19381 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 15:01:56.668203 19381 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 15:01:56.668251 19381 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 15:01:56.668279 19381 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 15:01:56.675441 19381 rpc_server.cc:307] RPC server started. Bound to: 127.13.75.130:44537
I20260923 15:01:56.675526 19509 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.75.130:44537 every 8 connection(s)
I20260923 15:01:56.675922 19381 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175691491353-13614-0/minicluster-data/ts-1/data/info.pb
Sep 23 15:01:56 dist-test-slave-cnb0 krb5kdc[19140](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175716, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.130@KRBTEST.COM for kudu/127.13.75.190@KRBTEST.COM
I20260923 15:01:56.685122 13614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKC4soy/build/release/bin/kudu as pid 19381
I20260923 15:01:56.685261 13614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175691491353-13614-0/minicluster-data/ts-1/wal/instance
I20260923 15:01:56.685473 13614 external_mini_cluster.cc:1428] Setting key 1028fbdaa46611877c3b1e1b9d38d22e
I20260923 15:01:56.687147 19224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:56.676912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.130:54831 (local address 127.13.75.190:39487)
0923 15:01:56.676982 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0923 15:01:56.676984 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:01:56.677187 (+ 203us) server_negotiation.cc:407] Connection header received
0923 15:01:56.678260 (+ 1073us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:01:56.678262 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:01:56.678279 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:01:56.678366 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:01:56.679184 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:56.679779 (+ 595us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:56.680973 (+ 1194us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:56.681203 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:56.681268 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:01:56.683862 (+ 2594us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:01:56.683863 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:01:56.683864 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:01:56.683871 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:01:56.685202 (+ 1331us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:01:56.685825 (+ 623us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:01:56.685826 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:01:56.685827 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:01:56.685906 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:01:56.686223 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:01:56.686224 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:01:56.686224 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:01:56.686472 (+ 248us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:01:56.686561 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0923 15:01:56.686847 (+ 286us) server_negotiation.cc:299] Negotiation successful
0923 15:01:56.686986 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260923 15:01:56.687180 19512 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:56.676810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:39487 (local address 127.13.75.130:54831)
0923 15:01:56.677032 (+ 222us) negotiation.cc:107] Waiting for socket to connect
0923 15:01:56.677096 (+ 64us) client_negotiation.cc:174] Beginning negotiation
0923 15:01:56.678089 (+ 993us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:01:56.678496 (+ 407us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:01:56.678500 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:01:56.678806 (+ 306us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:01:56.679093 (+ 287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:01:56.679099 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:01:56.679897 (+ 798us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:01:56.679898 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:01:56.680875 (+ 977us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:01:56.680878 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:01:56.681289 (+ 411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:01:56.681290 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:01:56.681397 (+ 107us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:01:56.682040 (+ 643us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:01:56.682049 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:01:56.683753 (+ 1704us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:01:56.685335 (+ 1582us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:01:56.685344 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:01:56.685347 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:01:56.685717 (+ 370us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:01:56.686002 (+ 285us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:01:56.686003 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:01:56.686003 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:01:56.686141 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:01:56.686559 (+ 418us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:01:56.686564 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:01:56.686694 (+ 130us) client_negotiation.cc:769] Sending connection context
0923 15:01:56.686786 (+ 92us) client_negotiation.cc:240] Negotiation successful
0923 15:01:56.686981 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":179,"thread_start_us":147,"threads_started":1}
I20260923 15:01:56.687736 19510 heartbeater.cc:347] Connected to a master server at 127.13.75.190:39487
I20260923 15:01:56.687851 19510 heartbeater.cc:464] Registering TS with master...
I20260923 15:01:56.688045 19510 heartbeater.cc:511] Master 127.13.75.190:39487 requested a full tablet report, sending...
I20260923 15:01:56.688540 19185 ts_manager.cc:194] Registered new tserver with Master: c329a2dfbf8242848b8e1a05139af590 (127.13.75.130:44537)
I20260923 15:01:56.689131 19185 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.75.130@KRBTEST.COM'} at 127.13.75.130:54831
WARNING: no policy specified for kudu/127.13.75.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.75.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.75.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.131.keytab.
Entry for principal kudu/127.13.75.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.131.keytab.
I20260923 15:01:56.718230 13614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKC4soy/build/release/bin/kudu
/tmp/dist-test-taskKC4soy/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175691491353-13614-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175691491353-13614-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175691491353-13614-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175691491353-13614-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.75.131:0
--local_ip_for_outbound_sockets=127.13.75.131
--webserver_interface=127.13.75.131
--webserver_port=0
--tserver_master_addrs=127.13.75.190:39487
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.131.keytab
--principal=kudu/127.13.75.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.75.148:44133
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175691491353-13614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175691491353-13614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175691491353-13614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:01:56.804903 19515 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:01:56.805130 19515 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:01:56.805155 19515 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:01:56.806284 19515 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:01:56.806318 19515 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:01:56.806345 19515 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.75.131
I20260923 15:01:56.807864 19515 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.75.148:44133
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175691491353-13614-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175691491353-13614-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.131.keytab
--principal=kudu/127.13.75.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.75.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175691491353-13614-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.75.131
--webserver_port=0
--tserver_master_addrs=127.13.75.190:39487
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19515
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.75.131
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175691491353-13614-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:01:56.808081 19515 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:01:56.808261 19515 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:01:56.813860 19520 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:01:56.813861 19521 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:01:56.814184 19515 server_base.cc:1034] running on GCE node
W20260923 15:01:56.813860 19523 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:01:56.814459 19515 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:01:56.814788 19515 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:01:56.816030 19515 hybrid_clock.cc:648] HybridClock initialized: now 1790175716816030 us; error 73 us; skew 500 ppm
Sep 23 15:01:56 dist-test-slave-cnb0 krb5kdc[19140](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175716, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:01:56.819159 19515 init.cc:377] Logged in from keytab as kudu/127.13.75.131@KRBTEST.COM (short username kudu)
I20260923 15:01:56.819813 19515 webserver.cc:460] Webserver started at http://127.13.75.131:33863/ using document root <none> and password file <none>
I20260923 15:01:56.820086 19515 fs_manager.cc:359] Metadata directory not provided
I20260923 15:01:56.820168 19515 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:01:56.820348 19515 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:01:56.821745 19515 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175691491353-13614-0/minicluster-data/ts-2/data/instance:
uuid: "407f24fb134e433a986813b0aadb702a"
format_stamp: "Formatted at 2026-09-23 15:01:56 on dist-test-slave-cnb0"
server_key: "a2ee1e5cfce988bc06e60319c40c0629"
server_key_iv: "29d13083cd1b62163669c7e1458a76db"
server_key_version: "encryptionkey@0"
I20260923 15:01:56.822171 19515 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175691491353-13614-0/minicluster-data/ts-2/wal/instance:
uuid: "407f24fb134e433a986813b0aadb702a"
format_stamp: "Formatted at 2026-09-23 15:01:56 on dist-test-slave-cnb0"
server_key: "a2ee1e5cfce988bc06e60319c40c0629"
server_key_iv: "29d13083cd1b62163669c7e1458a76db"
server_key_version: "encryptionkey@0"
I20260923 15:01:56.823868 19515 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260923 15:01:56.824895 19530 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:01:56.825210 19515 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:01:56.825299 19515 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175691491353-13614-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175691491353-13614-0/minicluster-data/ts-2/wal
uuid: "407f24fb134e433a986813b0aadb702a"
format_stamp: "Formatted at 2026-09-23 15:01:56 on dist-test-slave-cnb0"
server_key: "a2ee1e5cfce988bc06e60319c40c0629"
server_key_iv: "29d13083cd1b62163669c7e1458a76db"
server_key_version: "encryptionkey@0"
I20260923 15:01:56.825407 19515 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175691491353-13614-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175691491353-13614-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175691491353-13614-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:01:56.855775 19515 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:01:56.858270 19515 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:01:56.858394 19515 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:01:56.858647 19515 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 15:01:56.859002 19515 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 15:01:56.859046 19515 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 15:01:56.859071 19515 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 15:01:56.859082 19515 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 15:01:56.865337 19515 rpc_server.cc:307] RPC server started. Bound to: 127.13.75.131:33153
I20260923 15:01:56.865383 19643 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.75.131:33153 every 8 connection(s)
I20260923 15:01:56.865782 19515 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175691491353-13614-0/minicluster-data/ts-2/data/info.pb
Sep 23 15:01:56 dist-test-slave-cnb0 krb5kdc[19140](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175716, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.131@KRBTEST.COM for kudu/127.13.75.190@KRBTEST.COM
I20260923 15:01:56.874609 13614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKC4soy/build/release/bin/kudu as pid 19515
I20260923 15:01:56.874681 13614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175691491353-13614-0/minicluster-data/ts-2/wal/instance
I20260923 15:01:56.874867 13614 external_mini_cluster.cc:1428] Setting key 88c43476d6c3a2962ccc2933ee262c03
I20260923 15:01:56.877099 19224 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:56.866725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.131:39987 (local address 127.13.75.190:39487)
0923 15:01:56.866835 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0923 15:01:56.866837 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:01:56.867101 (+ 264us) server_negotiation.cc:407] Connection header received
0923 15:01:56.868147 (+ 1046us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:01:56.868148 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:01:56.868162 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:01:56.868210 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:01:56.868881 (+ 671us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:56.869391 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:56.870476 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:56.870680 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:56.870767 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:01:56.873355 (+ 2588us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:01:56.873356 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:01:56.873356 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:01:56.873365 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:01:56.875292 (+ 1927us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:01:56.875995 (+ 703us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:01:56.875996 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:01:56.875997 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:01:56.876067 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:01:56.876349 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:01:56.876350 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:01:56.876350 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:01:56.876507 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:01:56.876565 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0923 15:01:56.876963 (+ 398us) server_negotiation.cc:299] Negotiation successful
0923 15:01:56.877032 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":75}
I20260923 15:01:56.877360 19646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:56.866608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:39487 (local address 127.13.75.131:39987)
0923 15:01:56.866867 (+ 259us) negotiation.cc:107] Waiting for socket to connect
0923 15:01:56.866977 (+ 110us) client_negotiation.cc:174] Beginning negotiation
0923 15:01:56.868017 (+ 1040us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:01:56.868290 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:01:56.868295 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:01:56.868573 (+ 278us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:01:56.868768 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:01:56.868772 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:01:56.869492 (+ 720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:01:56.869493 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:01:56.870367 (+ 874us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:01:56.870370 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:01:56.870780 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:01:56.870781 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:01:56.870975 (+ 194us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:01:56.871746 (+ 771us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:01:56.871755 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:01:56.873237 (+ 1482us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:01:56.875432 (+ 2195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:01:56.875441 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:01:56.875443 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:01:56.875853 (+ 410us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:01:56.876145 (+ 292us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:01:56.876146 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:01:56.876147 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:01:56.876282 (+ 135us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:01:56.876594 (+ 312us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:01:56.876600 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:01:56.876765 (+ 165us) client_negotiation.cc:769] Sending connection context
0923 15:01:56.876934 (+ 169us) client_negotiation.cc:240] Negotiation successful
0923 15:01:56.877074 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":106,"threads_started":1}
I20260923 15:01:56.878036 19644 heartbeater.cc:347] Connected to a master server at 127.13.75.190:39487
I20260923 15:01:56.878154 19644 heartbeater.cc:464] Registering TS with master...
I20260923 15:01:56.878337 19644 heartbeater.cc:511] Master 127.13.75.190:39487 requested a full tablet report, sending...
I20260923 15:01:56.878727 19185 ts_manager.cc:194] Registered new tserver with Master: 407f24fb134e433a986813b0aadb702a (127.13.75.131:33153)
I20260923 15:01:56.879185 19185 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.75.131@KRBTEST.COM'} at 127.13.75.131:39987
I20260923 15:01:56.887722 13614 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal test-admin with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175691491353-13614-0/krb5kdc/test-admin.keytab.
Entry for principal test-admin with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175691491353-13614-0/krb5kdc/test-admin.keytab.
Sep 23 15:01:56 dist-test-slave-cnb0 krb5kdc[19140](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175716, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:01:56.908413 13614 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
W20260923 15:01:56.910409 19651 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 15:01:56.909459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40450 (local address 127.13.75.129:44681)
0923 15:01:56.909689 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0923 15:01:56.909695 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 15:01:56.909710 (+ 15us) server_negotiation.cc:407] Connection header received
0923 15:01:56.909742 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:01:56.909744 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:01:56.909757 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:01:56.909828 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:01:56.910326 (+ 498us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40450: BlockingRecv error: recv got EOF from 127.0.0.1:40450 (error 108)
Metrics: {"server-negotiator.queue_time_us":194,"thread_start_us":154,"threads_started":1}
W20260923 15:01:56.910585 13614 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175691491353-13614-0/krb5kdc/krb5cc' of type FILE
Sep 23 15:01:56 dist-test-slave-cnb0 krb5kdc[19140](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175716, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:01:56.911690 13614 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 23 15:01:56 dist-test-slave-cnb0 krb5kdc[19140](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175716, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:01:56.912626 13614 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 23 15:01:56 dist-test-slave-cnb0 krb5kdc[19140](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175716, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.75.129@KRBTEST.COM
I20260923 15:01:56.923166 19651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:56.913238 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40466 (local address 127.13.75.129:44681)
0923 15:01:56.913340 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0923 15:01:56.913343 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:01:56.913506 (+ 163us) server_negotiation.cc:407] Connection header received
0923 15:01:56.913634 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:01:56.913635 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:01:56.913650 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:01:56.913737 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:01:56.914207 (+ 470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:56.915254 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:56.916216 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:56.916636 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:56.916738 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:01:56.919654 (+ 2916us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:01:56.919671 (+ 17us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:01:56.919671 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:01:56.919686 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:01:56.921703 (+ 2017us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:01:56.922199 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:01:56.922205 (+ 6us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:01:56.922206 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:01:56.922272 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:01:56.922506 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:01:56.922507 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:01:56.922507 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:01:56.922670 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:01:56.922835 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0923 15:01:56.922947 (+ 112us) server_negotiation.cc:299] Negotiation successful
0923 15:01:56.923041 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
W20260923 15:01:56.925765 19651 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 15:01:56.925005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40470 (local address 127.13.75.129:44681)
0923 15:01:56.925102 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0923 15:01:56.925105 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:01:56.925201 (+ 96us) server_negotiation.cc:407] Connection header received
0923 15:01:56.925309 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:01:56.925310 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:01:56.925322 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:01:56.925390 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:01:56.925704 (+ 314us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40470: BlockingRecv error: recv got EOF from 127.0.0.1:40470 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
W20260923 15:01:56.925961 13614 init.cc:244] Error while opening credential cache '/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175691491353-13614-0/krb5kdc/krb5cc' of type FILE: Runtime error: End of credential cache reached (filename: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175691491353-13614-0/krb5kdc/krb5cc)
Sep 23 15:01:56 dist-test-slave-cnb0 krb5kdc[19140](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175716, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:01:56.927091 13614 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 23 15:01:56 dist-test-slave-cnb0 krb5kdc[19140](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175716, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:01:56.928009 13614 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 23 15:01:56 dist-test-slave-cnb0 krb5kdc[19140](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175716, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.75.129@KRBTEST.COM
I20260923 15:01:56.935503 19651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:56.928672 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40474 (local address 127.13.75.129:44681)
0923 15:01:56.928742 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0923 15:01:56.928745 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:01:56.928803 (+ 58us) server_negotiation.cc:407] Connection header received
0923 15:01:56.928832 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:01:56.928833 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:01:56.928845 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:01:56.928903 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:01:56.929273 (+ 370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:56.929969 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:56.930701 (+ 732us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:56.930998 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:56.931079 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:01:56.933113 (+ 2034us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:01:56.933114 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:01:56.933114 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:01:56.933121 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:01:56.934333 (+ 1212us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:01:56.934673 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:01:56.934674 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:01:56.934674 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:01:56.934725 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:01:56.934981 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:01:56.934982 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:01:56.934983 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:01:56.935091 (+ 108us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:01:56.935170 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0923 15:01:56.935280 (+ 110us) server_negotiation.cc:299] Negotiation successful
0923 15:01:56.935343 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
W20260923 15:01:56.937220 19651 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 15:01:56.936658 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40480 (local address 127.13.75.129:44681)
0923 15:01:56.936694 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0923 15:01:56.936696 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:01:56.936741 (+ 45us) server_negotiation.cc:407] Connection header received
0923 15:01:56.936790 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:01:56.936791 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:01:56.936799 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:01:56.936845 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:01:56.937155 (+ 310us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40480: BlockingRecv error: recv got EOF from 127.0.0.1:40480 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260923 15:01:56.937307 13614 init.cc:300] Could not find kerberos principal in credential cache '/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestCorruptKerberosCC.1790175691491353-13614-0/krb5kdc/krb5cc' of type FILE
Sep 23 15:01:56 dist-test-slave-cnb0 krb5kdc[19140](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175716, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:01:56.938297 13614 init.cc:377] Logged in from keytab as test-admin@KRBTEST.COM (short username test-admin)
Sep 23 15:01:56 dist-test-slave-cnb0 krb5kdc[19140](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175716, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:01:56.939200 13614 init.cc:295] Successfully reacquired a new kerberos TGT
Sep 23 15:01:56 dist-test-slave-cnb0 krb5kdc[19140](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175716, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.75.129@KRBTEST.COM
I20260923 15:01:56.946159 19651 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:56.939738 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40490 (local address 127.13.75.129:44681)
0923 15:01:56.939798 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0923 15:01:56.939800 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:01:56.939865 (+ 65us) server_negotiation.cc:407] Connection header received
0923 15:01:56.939928 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:01:56.939929 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:01:56.939938 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:01:56.940035 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:01:56.940396 (+ 361us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:56.940899 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:56.941599 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:56.941786 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:56.941877 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:01:56.943874 (+ 1997us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:01:56.943875 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:01:56.943875 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:01:56.943882 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:01:56.945248 (+ 1366us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:01:56.945576 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:01:56.945577 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:01:56.945577 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:01:56.945623 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:01:56.945781 (+ 158us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:01:56.945781 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:01:56.945781 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:01:56.945866 (+ 85us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:01:56.945934 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0923 15:01:56.946044 (+ 110us) server_negotiation.cc:299] Negotiation successful
0923 15:01:56.946096 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260923 15:01:56.947000 13614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKC4soy/build/release/bin/kudu with pid 19247
I20260923 15:01:56.952529 13614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKC4soy/build/release/bin/kudu with pid 19381
I20260923 15:01:56.957019 13614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKC4soy/build/release/bin/kudu with pid 19515
I20260923 15:01:56.961431 13614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKC4soy/build/release/bin/kudu with pid 19153
2026-09-23T15:01:56Z chronyd exiting
[ OK ] SecurityITest.TestCorruptKerberosCC (2002 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipalMultipleMaster
Loading random data
Initializing database '/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 15:01:57 dist-test-slave-cnb0 krb5kdc[19664](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 15:01:57 dist-test-slave-cnb0 krb5kdc[19664](info): set up 2 sockets
Sep 23 15:01:57 dist-test-slave-cnb0 krb5kdc[19664](info): commencing operation
krb5kdc: starting...
W20260923 15:01:58.052729 13614 mini_kdc.cc:121] Time spent starting KDC: real 1.070s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/krb5kdc/test-user.keytab.
Sep 23 15:01:58 dist-test-slave-cnb0 krb5kdc[19664](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175718, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T15:01:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T15:01:58Z Disabled control of system clock
WARNING: no policy specified for oryx/127.13.75.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.13.75.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.13.75.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/krb5kdc/oryx_127.13.75.190.keytab.
Entry for principal oryx/127.13.75.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/krb5kdc/oryx_127.13.75.190.keytab.
I20260923 15:01:58.185937 13614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKC4soy/build/release/bin/kudu
/tmp/dist-test-taskKC4soy/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.75.190:35255
--webserver_interface=127.13.75.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/krb5kdc/oryx_127.13.75.190.keytab
--principal=oryx/127.13.75.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.75.148:36053
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.75.190:35255
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:01:58.270129 19677 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:01:58.270423 19677 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:01:58.270449 19677 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:01:58.271818 19677 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 15:01:58.271869 19677 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:01:58.271884 19677 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:01:58.271898 19677 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 15:01:58.271910 19677 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 15:01:58.273407 19677 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.75.148:36053
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.75.190:35255
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/krb5kdc/oryx_127.13.75.190.keytab
--principal=oryx/127.13.75.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.75.190:35255
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.75.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19677
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:01:58.273622 19677 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:01:58.273811 19677 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:01:58.279084 19682 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:01:58.279079 19683 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:01:58.279106 19677 server_base.cc:1034] running on GCE node
W20260923 15:01:58.279084 19685 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:01:58.279668 19677 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:01:58.280033 19677 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:01:58.281225 19677 hybrid_clock.cc:648] HybridClock initialized: now 1790175718281199 us; error 46 us; skew 500 ppm
Sep 23 15:01:58 dist-test-slave-cnb0 krb5kdc[19664](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175718, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.75.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:01:58.287716 19677 init.cc:377] Logged in from keytab as oryx/127.13.75.190@KRBTEST.COM (short username oryx)
I20260923 15:01:58.288244 19677 webserver.cc:460] Webserver started at http://127.13.75.190:42327/ using document root <none> and password file <none>
I20260923 15:01:58.288530 19677 fs_manager.cc:359] Metadata directory not provided
I20260923 15:01:58.288610 19677 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:01:58.288771 19677 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:01:58.289922 19677 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/minicluster-data/master-0/data/instance:
uuid: "176378cdc0584ce69eca703063507558"
format_stamp: "Formatted at 2026-09-23 15:01:58 on dist-test-slave-cnb0"
server_key: "b0c6fa45bf85901e2572038bf18c2c7e"
server_key_iv: "a1036f7a0fbcc2662ce3c394af0d3360"
server_key_version: "encryptionkey@0"
I20260923 15:01:58.290304 19677 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/minicluster-data/master-0/wal/instance:
uuid: "176378cdc0584ce69eca703063507558"
format_stamp: "Formatted at 2026-09-23 15:01:58 on dist-test-slave-cnb0"
server_key: "b0c6fa45bf85901e2572038bf18c2c7e"
server_key_iv: "a1036f7a0fbcc2662ce3c394af0d3360"
server_key_version: "encryptionkey@0"
I20260923 15:01:58.291963 19677 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 15:01:58.292903 19692 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:01:58.293128 19677 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:01:58.293251 19677 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/minicluster-data/master-0/data,/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/minicluster-data/master-0/wal
uuid: "176378cdc0584ce69eca703063507558"
format_stamp: "Formatted at 2026-09-23 15:01:58 on dist-test-slave-cnb0"
server_key: "b0c6fa45bf85901e2572038bf18c2c7e"
server_key_iv: "a1036f7a0fbcc2662ce3c394af0d3360"
server_key_version: "encryptionkey@0"
I20260923 15:01:58.293339 19677 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:01:58.322291 19677 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:01:58.324555 19677 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:01:58.324738 19677 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:01:58.328907 19677 rpc_server.cc:307] RPC server started. Bound to: 127.13.75.190:35255
I20260923 15:01:58.328940 19744 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.75.190:35255 every 8 connection(s)
I20260923 15:01:58.329360 19677 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/minicluster-data/master-0/data/info.pb
I20260923 15:01:58.330401 19745 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:01:58.331813 13614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKC4soy/build/release/bin/kudu as pid 19677
I20260923 15:01:58.331936 13614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/minicluster-data/master-0/wal/instance
I20260923 15:01:58.332152 13614 external_mini_cluster.cc:1428] Setting key 9aecd06f95afba340f5829a1dba60654
I20260923 15:01:58.333127 19745 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 176378cdc0584ce69eca703063507558: Bootstrap starting.
I20260923 15:01:58.333734 19745 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 176378cdc0584ce69eca703063507558: Neither blocks nor log segments found. Creating new log.
I20260923 15:01:58.334039 19745 log.cc:824] T 00000000000000000000000000000000 P 176378cdc0584ce69eca703063507558: Log is configured to *not* fsync() on all Append() calls
I20260923 15:01:58.335016 19745 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 176378cdc0584ce69eca703063507558: No bootstrap required, opened a new log
I20260923 15:01:58.337623 19745 raft_consensus.cc:348] T 00000000000000000000000000000000 P 176378cdc0584ce69eca703063507558 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "176378cdc0584ce69eca703063507558" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 35255 } }
I20260923 15:01:58.337831 19745 raft_consensus.cc:374] T 00000000000000000000000000000000 P 176378cdc0584ce69eca703063507558 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:01:58.337886 19745 raft_consensus.cc:729] T 00000000000000000000000000000000 P 176378cdc0584ce69eca703063507558 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 176378cdc0584ce69eca703063507558, State: Initialized, Role: FOLLOWER
I20260923 15:01:58.338083 19745 consensus_queue.cc:260] T 00000000000000000000000000000000 P 176378cdc0584ce69eca703063507558 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "176378cdc0584ce69eca703063507558" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 35255 } }
I20260923 15:01:58.338217 19745 raft_consensus.cc:388] T 00000000000000000000000000000000 P 176378cdc0584ce69eca703063507558 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 15:01:58.338266 19745 raft_consensus.cc:482] T 00000000000000000000000000000000 P 176378cdc0584ce69eca703063507558 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 15:01:58.338321 19745 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 176378cdc0584ce69eca703063507558 [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:01:58.339099 19745 raft_consensus.cc:504] T 00000000000000000000000000000000 P 176378cdc0584ce69eca703063507558 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "176378cdc0584ce69eca703063507558" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 35255 } }
I20260923 15:01:58.339201 19745 leader_election.cc:302] T 00000000000000000000000000000000 P 176378cdc0584ce69eca703063507558 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 176378cdc0584ce69eca703063507558; no voters:
I20260923 15:01:58.339434 19745 leader_election.cc:288] T 00000000000000000000000000000000 P 176378cdc0584ce69eca703063507558 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 15:01:58.339497 19750 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 176378cdc0584ce69eca703063507558 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 15:01:58.339684 19750 raft_consensus.cc:686] T 00000000000000000000000000000000 P 176378cdc0584ce69eca703063507558 [term 1 LEADER]: Becoming Leader. State: Replica: 176378cdc0584ce69eca703063507558, State: Running, Role: LEADER
I20260923 15:01:58.339769 19745 sys_catalog.cc:564] T 00000000000000000000000000000000 P 176378cdc0584ce69eca703063507558 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 15:01:58.339802 19750 consensus_queue.cc:237] T 00000000000000000000000000000000 P 176378cdc0584ce69eca703063507558 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "176378cdc0584ce69eca703063507558" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 35255 } }
I20260923 15:01:58.340161 19750 sys_catalog.cc:455] T 00000000000000000000000000000000 P 176378cdc0584ce69eca703063507558 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 176378cdc0584ce69eca703063507558. Latest consensus state: current_term: 1 leader_uuid: "176378cdc0584ce69eca703063507558" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "176378cdc0584ce69eca703063507558" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 35255 } } }
I20260923 15:01:58.340260 19750 sys_catalog.cc:458] T 00000000000000000000000000000000 P 176378cdc0584ce69eca703063507558 [sys.catalog]: This master's current role is: LEADER
I20260923 15:01:58.340169 19752 sys_catalog.cc:455] T 00000000000000000000000000000000 P 176378cdc0584ce69eca703063507558 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "176378cdc0584ce69eca703063507558" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "176378cdc0584ce69eca703063507558" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 35255 } } }
I20260923 15:01:58.340485 19752 sys_catalog.cc:458] T 00000000000000000000000000000000 P 176378cdc0584ce69eca703063507558 [sys.catalog]: This master's current role is: LEADER
Sep 23 15:01:58 dist-test-slave-cnb0 krb5kdc[19664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175718, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.13.75.190@KRBTEST.COM
W20260923 15:01:58.341786 19765 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 176378cdc0584ce69eca703063507558: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260923 15:01:58.341846 19765 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260923 15:01:58.342044 19756 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 15:01:58.342218 19756 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 15:01:58.344200 19756 catalog_manager.cc:1347] Generated new cluster ID: 4b5be46274e44cd197c2c043e352da13
I20260923 15:01:58.344275 19756 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 15:01:58.345291 19748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:58.332760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50766 (local address 127.13.75.190:35255)
0923 15:01:58.332954 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0923 15:01:58.332962 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0923 15:01:58.332990 (+ 28us) server_negotiation.cc:407] Connection header received
0923 15:01:58.333447 (+ 457us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:01:58.333454 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:01:58.333519 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:01:58.333676 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:01:58.334475 (+ 799us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:58.336144 (+ 1669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:58.336949 (+ 805us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:58.337719 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:58.337848 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:01:58.341216 (+ 3368us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:01:58.341221 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:01:58.341222 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:01:58.341238 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:01:58.343425 (+ 2187us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:01:58.343884 (+ 459us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:01:58.343888 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:01:58.343888 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:01:58.344013 (+ 125us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:01:58.344277 (+ 264us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:01:58.344279 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:01:58.344279 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:01:58.344617 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:01:58.344779 (+ 162us) server_negotiation.cc:1035] Waiting for connection context
0923 15:01:58.344939 (+ 160us) server_negotiation.cc:299] Negotiation successful
0923 15:01:58.345086 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":83,"threads_started":1}
I20260923 15:01:58.351801 19756 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 15:01:58.352453 19756 catalog_manager.cc:1504] Loading token signing keys...
I20260923 15:01:58.363365 19756 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 176378cdc0584ce69eca703063507558: Generated new TSK 0
I20260923 15:01:58.363624 19756 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.13.75.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.13.75.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.13.75.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/krb5kdc/oryx_127.13.75.129.keytab.
Entry for principal oryx/127.13.75.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/krb5kdc/oryx_127.13.75.129.keytab.
I20260923 15:01:58.394274 13614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKC4soy/build/release/bin/kudu
/tmp/dist-test-taskKC4soy/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.75.129:0
--local_ip_for_outbound_sockets=127.13.75.129
--webserver_interface=127.13.75.129
--webserver_port=0
--tserver_master_addrs=127.13.75.190:35255
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/krb5kdc/oryx_127.13.75.129.keytab
--principal=oryx/127.13.75.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.75.148:36053
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:01:58.478008 19771 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:01:58.478217 19771 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:01:58.478242 19771 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:01:58.479533 19771 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:01:58.479576 19771 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:01:58.479604 19771 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.75.129
I20260923 15:01:58.481096 19771 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.75.148:36053
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/krb5kdc/oryx_127.13.75.129.keytab
--principal=oryx/127.13.75.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.75.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.75.129
--webserver_port=0
--tserver_master_addrs=127.13.75.190:35255
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19771
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.75.129
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:01:58.481354 19771 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:01:58.481604 19771 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:01:58.486757 19776 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:01:58.486826 19771 server_base.cc:1034] running on GCE node
W20260923 15:01:58.486749 19777 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:01:58.486752 19779 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:01:58.487236 19771 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:01:58.487558 19771 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:01:58.488746 19771 hybrid_clock.cc:648] HybridClock initialized: now 1790175718488729 us; error 49 us; skew 500 ppm
Sep 23 15:01:58 dist-test-slave-cnb0 krb5kdc[19664](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175718, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.75.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:01:58.491559 19771 init.cc:377] Logged in from keytab as oryx/127.13.75.129@KRBTEST.COM (short username oryx)
I20260923 15:01:58.492058 19771 webserver.cc:460] Webserver started at http://127.13.75.129:36225/ using document root <none> and password file <none>
I20260923 15:01:58.492312 19771 fs_manager.cc:359] Metadata directory not provided
I20260923 15:01:58.492384 19771 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:01:58.492542 19771 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:01:58.493615 19771 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/minicluster-data/ts-0/data/instance:
uuid: "e9ef8b623efa4412ae2bfff361a0a8a8"
format_stamp: "Formatted at 2026-09-23 15:01:58 on dist-test-slave-cnb0"
server_key: "69cfdf9c2f665a3d6ab4197dc11358d8"
server_key_iv: "0da10c81dae47b6d40a3ea133babe112"
server_key_version: "encryptionkey@0"
I20260923 15:01:58.493976 19771 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/minicluster-data/ts-0/wal/instance:
uuid: "e9ef8b623efa4412ae2bfff361a0a8a8"
format_stamp: "Formatted at 2026-09-23 15:01:58 on dist-test-slave-cnb0"
server_key: "69cfdf9c2f665a3d6ab4197dc11358d8"
server_key_iv: "0da10c81dae47b6d40a3ea133babe112"
server_key_version: "encryptionkey@0"
I20260923 15:01:58.495502 19771 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260923 15:01:58.496385 19786 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:01:58.496644 19771 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260923 15:01:58.496770 19771 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/minicluster-data/ts-0/wal
uuid: "e9ef8b623efa4412ae2bfff361a0a8a8"
format_stamp: "Formatted at 2026-09-23 15:01:58 on dist-test-slave-cnb0"
server_key: "69cfdf9c2f665a3d6ab4197dc11358d8"
server_key_iv: "0da10c81dae47b6d40a3ea133babe112"
server_key_version: "encryptionkey@0"
I20260923 15:01:58.496863 19771 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:01:58.508790 19771 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:01:58.511758 19771 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:01:58.511951 19771 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:01:58.512214 19771 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 15:01:58.512547 19771 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 15:01:58.512616 19771 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 15:01:58.512661 19771 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 15:01:58.512704 19771 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 15:01:58.518780 19771 rpc_server.cc:307] RPC server started. Bound to: 127.13.75.129:42411
I20260923 15:01:58.518819 19899 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.75.129:42411 every 8 connection(s)
I20260923 15:01:58.519263 19771 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/minicluster-data/ts-0/data/info.pb
I20260923 15:01:58.519955 13614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKC4soy/build/release/bin/kudu as pid 19771
I20260923 15:01:58.520072 13614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/minicluster-data/ts-0/wal/instance
I20260923 15:01:58.520313 13614 external_mini_cluster.cc:1428] Setting key 43e5f5b6054c7017409e3357eb3972f2
Sep 23 15:01:58 dist-test-slave-cnb0 krb5kdc[19664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175718, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.75.129@KRBTEST.COM for oryx/127.13.75.190@KRBTEST.COM
WARNING: no policy specified for oryx/127.13.75.130@KRBTEST.COM; defaulting to no policy
I20260923 15:01:58.532042 19748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:58.520189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.129:51663 (local address 127.13.75.190:35255)
0923 15:01:58.520273 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0923 15:01:58.520275 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:01:58.520553 (+ 278us) server_negotiation.cc:407] Connection header received
0923 15:01:58.521338 (+ 785us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:01:58.521340 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:01:58.521355 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:01:58.521425 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:01:58.523018 (+ 1593us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:58.523649 (+ 631us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:58.524768 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:58.525014 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:58.525117 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:01:58.528306 (+ 3189us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:01:58.528308 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:01:58.528308 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:01:58.528319 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:01:58.530257 (+ 1938us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:01:58.530759 (+ 502us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:01:58.530760 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:01:58.530760 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:01:58.530854 (+ 94us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:01:58.531291 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:01:58.531292 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:01:58.531293 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:01:58.531427 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:01:58.531513 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0923 15:01:58.531826 (+ 313us) server_negotiation.cc:299] Negotiation successful
0923 15:01:58.531931 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":50}
I20260923 15:01:58.532284 19902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:58.520137 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:35255 (local address 127.13.75.129:51663)
0923 15:01:58.520358 (+ 221us) negotiation.cc:107] Waiting for socket to connect
0923 15:01:58.520419 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0923 15:01:58.521170 (+ 751us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:01:58.521545 (+ 375us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:01:58.521553 (+ 8us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:01:58.522504 (+ 951us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:01:58.522849 (+ 345us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:01:58.522857 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:01:58.523752 (+ 895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:01:58.523753 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:01:58.524645 (+ 892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:01:58.524647 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:01:58.525131 (+ 484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:01:58.525132 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:01:58.525282 (+ 150us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:01:58.525960 (+ 678us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:01:58.525974 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:01:58.528153 (+ 2179us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:01:58.530361 (+ 2208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:01:58.530372 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:01:58.530374 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:01:58.530656 (+ 282us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:01:58.531023 (+ 367us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:01:58.531025 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:01:58.531025 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:01:58.531195 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:01:58.531504 (+ 309us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:01:58.531520 (+ 16us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:01:58.531683 (+ 163us) client_negotiation.cc:769] Sending connection context
0923 15:01:58.531804 (+ 121us) client_negotiation.cc:240] Negotiation successful
0923 15:01:58.531988 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":97,"threads_started":1}
I20260923 15:01:58.533043 19900 heartbeater.cc:347] Connected to a master server at 127.13.75.190:35255
I20260923 15:01:58.533176 19900 heartbeater.cc:464] Registering TS with master...
I20260923 15:01:58.533375 19900 heartbeater.cc:511] Master 127.13.75.190:35255 requested a full tablet report, sending...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.13.75.130@KRBTEST.COM" created.
I20260923 15:01:58.534150 19709 ts_manager.cc:194] Registered new tserver with Master: e9ef8b623efa4412ae2bfff361a0a8a8 (127.13.75.129:42411)
I20260923 15:01:58.535135 19709 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.13.75.129@KRBTEST.COM'} at 127.13.75.129:51663
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.13.75.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/krb5kdc/oryx_127.13.75.130.keytab.
Entry for principal oryx/127.13.75.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/krb5kdc/oryx_127.13.75.130.keytab.
I20260923 15:01:58.551373 13614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKC4soy/build/release/bin/kudu
/tmp/dist-test-taskKC4soy/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.75.130:0
--local_ip_for_outbound_sockets=127.13.75.130
--webserver_interface=127.13.75.130
--webserver_port=0
--tserver_master_addrs=127.13.75.190:35255
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/krb5kdc/oryx_127.13.75.130.keytab
--principal=oryx/127.13.75.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.75.148:36053
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:01:58.634616 19905 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:01:58.634891 19905 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:01:58.634977 19905 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:01:58.636222 19905 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:01:58.636305 19905 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:01:58.636353 19905 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.75.130
I20260923 15:01:58.637836 19905 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.75.148:36053
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/krb5kdc/oryx_127.13.75.130.keytab
--principal=oryx/127.13.75.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.75.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.75.130
--webserver_port=0
--tserver_master_addrs=127.13.75.190:35255
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.19905
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.75.130
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:01:58.638134 19905 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:01:58.638382 19905 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:01:58.644634 19913 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:01:58.644717 19905 server_base.cc:1034] running on GCE node
W20260923 15:01:58.644624 19910 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:01:58.644624 19911 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:01:58.645112 19905 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:01:58.645378 19905 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:01:58.646536 19905 hybrid_clock.cc:648] HybridClock initialized: now 1790175718646524 us; error 30 us; skew 500 ppm
Sep 23 15:01:58 dist-test-slave-cnb0 krb5kdc[19664](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175718, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.75.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:01:58.649165 19905 init.cc:377] Logged in from keytab as oryx/127.13.75.130@KRBTEST.COM (short username oryx)
I20260923 15:01:58.649667 19905 webserver.cc:460] Webserver started at http://127.13.75.130:44197/ using document root <none> and password file <none>
I20260923 15:01:58.649902 19905 fs_manager.cc:359] Metadata directory not provided
I20260923 15:01:58.649947 19905 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:01:58.650128 19905 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:01:58.651321 19905 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/minicluster-data/ts-1/data/instance:
uuid: "a32e8b9f779e4b9288aafcdb2e4a1c7a"
format_stamp: "Formatted at 2026-09-23 15:01:58 on dist-test-slave-cnb0"
server_key: "d76b0efe1e2c1e91807e818dbce81b79"
server_key_iv: "c17c11f711d622f9f0d278b7220ac308"
server_key_version: "encryptionkey@0"
I20260923 15:01:58.651696 19905 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/minicluster-data/ts-1/wal/instance:
uuid: "a32e8b9f779e4b9288aafcdb2e4a1c7a"
format_stamp: "Formatted at 2026-09-23 15:01:58 on dist-test-slave-cnb0"
server_key: "d76b0efe1e2c1e91807e818dbce81b79"
server_key_iv: "c17c11f711d622f9f0d278b7220ac308"
server_key_version: "encryptionkey@0"
I20260923 15:01:58.653070 19905 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260923 15:01:58.653896 19920 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:01:58.654094 19905 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:01:58.654201 19905 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/minicluster-data/ts-1/wal
uuid: "a32e8b9f779e4b9288aafcdb2e4a1c7a"
format_stamp: "Formatted at 2026-09-23 15:01:58 on dist-test-slave-cnb0"
server_key: "d76b0efe1e2c1e91807e818dbce81b79"
server_key_iv: "c17c11f711d622f9f0d278b7220ac308"
server_key_version: "encryptionkey@0"
I20260923 15:01:58.654287 19905 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:01:58.671597 19905 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:01:58.673928 19905 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:01:58.674151 19905 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:01:58.674453 19905 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 15:01:58.674871 19905 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 15:01:58.675002 19905 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 15:01:58.675051 19905 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 15:01:58.675083 19905 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 15:01:58.681958 19905 rpc_server.cc:307] RPC server started. Bound to: 127.13.75.130:42895
I20260923 15:01:58.682405 19905 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/minicluster-data/ts-1/data/info.pb
I20260923 15:01:58.682719 20033 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.75.130:42895 every 8 connection(s)
I20260923 15:01:58.687004 13614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKC4soy/build/release/bin/kudu as pid 19905
I20260923 15:01:58.687111 13614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/minicluster-data/ts-1/wal/instance
I20260923 15:01:58.687315 13614 external_mini_cluster.cc:1428] Setting key fd4124d4340634bbaa54aba796c23153
Sep 23 15:01:58 dist-test-slave-cnb0 krb5kdc[19664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175718, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.75.130@KRBTEST.COM for oryx/127.13.75.190@KRBTEST.COM
I20260923 15:01:58.693778 19748 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:58.683322 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.130:58099 (local address 127.13.75.190:35255)
0923 15:01:58.683396 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0923 15:01:58.683398 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:01:58.683652 (+ 254us) server_negotiation.cc:407] Connection header received
0923 15:01:58.684308 (+ 656us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:01:58.684310 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:01:58.684321 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:01:58.684387 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:01:58.685419 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:58.685938 (+ 519us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:58.687256 (+ 1318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:58.687626 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:58.687719 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:01:58.690595 (+ 2876us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:01:58.690596 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:01:58.690597 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:01:58.690605 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:01:58.692194 (+ 1589us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:01:58.692690 (+ 496us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:01:58.692692 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:01:58.692692 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:01:58.692757 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:01:58.693062 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:01:58.693063 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:01:58.693064 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:01:58.693198 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:01:58.693280 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0923 15:01:58.693582 (+ 302us) server_negotiation.cc:299] Negotiation successful
0923 15:01:58.693681 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260923 15:01:58.694062 20036 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:58.683309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:35255 (local address 127.13.75.130:58099)
0923 15:01:58.683499 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0923 15:01:58.683549 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0923 15:01:58.684178 (+ 629us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:01:58.684448 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:01:58.684452 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:01:58.685105 (+ 653us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:01:58.685320 (+ 215us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:01:58.685325 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:01:58.686039 (+ 714us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:01:58.686040 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:01:58.687107 (+ 1067us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:01:58.687110 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:01:58.687733 (+ 623us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:01:58.687734 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:01:58.687871 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:01:58.688505 (+ 634us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:01:58.688516 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:01:58.690490 (+ 1974us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:01:58.692307 (+ 1817us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:01:58.692316 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:01:58.692318 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:01:58.692604 (+ 286us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:01:58.692832 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:01:58.692833 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:01:58.692833 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:01:58.692974 (+ 141us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:01:58.693270 (+ 296us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:01:58.693277 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:01:58.693432 (+ 155us) client_negotiation.cc:769] Sending connection context
0923 15:01:58.693554 (+ 122us) client_negotiation.cc:240] Negotiation successful
0923 15:01:58.693737 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":113,"threads_started":1}
I20260923 15:01:58.694541 20034 heartbeater.cc:347] Connected to a master server at 127.13.75.190:35255
I20260923 15:01:58.694643 20034 heartbeater.cc:464] Registering TS with master...
I20260923 15:01:58.694859 20034 heartbeater.cc:511] Master 127.13.75.190:35255 requested a full tablet report, sending...
I20260923 15:01:58.695341 19709 ts_manager.cc:194] Registered new tserver with Master: a32e8b9f779e4b9288aafcdb2e4a1c7a (127.13.75.130:42895)
I20260923 15:01:58.695923 19709 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.13.75.130@KRBTEST.COM'} at 127.13.75.130:58099
WARNING: no policy specified for oryx/127.13.75.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.13.75.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.13.75.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/krb5kdc/oryx_127.13.75.131.keytab.
Entry for principal oryx/127.13.75.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/krb5kdc/oryx_127.13.75.131.keytab.
W20260923 15:01:59.302681 13614 mini_kdc.cc:262] Time spent creating service keytab for oryx/127.13.75.131: real 0.614s user 0.003s sys 0.000s
I20260923 15:01:59.303100 13614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKC4soy/build/release/bin/kudu
/tmp/dist-test-taskKC4soy/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.75.131:0
--local_ip_for_outbound_sockets=127.13.75.131
--webserver_interface=127.13.75.131
--webserver_port=0
--tserver_master_addrs=127.13.75.190:35255
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/krb5kdc/oryx_127.13.75.131.keytab
--principal=oryx/127.13.75.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.75.148:36053
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:01:59.388779 20039 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:01:59.389041 20039 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:01:59.389070 20039 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:01:59.390264 20039 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:01:59.390311 20039 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:01:59.390338 20039 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.75.131
I20260923 15:01:59.391861 20039 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.75.148:36053
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/krb5kdc/oryx_127.13.75.131.keytab
--principal=oryx/127.13.75.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.75.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.75.131
--webserver_port=0
--tserver_master_addrs=127.13.75.190:35255
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20039
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.75.131
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:01:59.392073 20039 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:01:59.392262 20039 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:01:59.397917 20044 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:01:59.398124 20047 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:01:59.398000 20039 server_base.cc:1034] running on GCE node
W20260923 15:01:59.398000 20045 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:01:59.398589 20039 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:01:59.398969 20039 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:01:59.400244 20039 hybrid_clock.cc:648] HybridClock initialized: now 1790175719400220 us; error 66 us; skew 500 ppm
Sep 23 15:01:59 dist-test-slave-cnb0 krb5kdc[19664](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175719, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.75.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:01:59.403327 20039 init.cc:377] Logged in from keytab as oryx/127.13.75.131@KRBTEST.COM (short username oryx)
I20260923 15:01:59.403884 20039 webserver.cc:460] Webserver started at http://127.13.75.131:36865/ using document root <none> and password file <none>
I20260923 15:01:59.404170 20039 fs_manager.cc:359] Metadata directory not provided
I20260923 15:01:59.404223 20039 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:01:59.404367 20039 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:01:59.405542 20039 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/minicluster-data/ts-2/data/instance:
uuid: "430f9682c1fe49f7b9fc238400127887"
format_stamp: "Formatted at 2026-09-23 15:01:59 on dist-test-slave-cnb0"
server_key: "938d7bf162a733df869127699b1592af"
server_key_iv: "e29dabf2097eefbf2db97bbbe59360a0"
server_key_version: "encryptionkey@0"
I20260923 15:01:59.405974 20039 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/minicluster-data/ts-2/wal/instance:
uuid: "430f9682c1fe49f7b9fc238400127887"
format_stamp: "Formatted at 2026-09-23 15:01:59 on dist-test-slave-cnb0"
server_key: "938d7bf162a733df869127699b1592af"
server_key_iv: "e29dabf2097eefbf2db97bbbe59360a0"
server_key_version: "encryptionkey@0"
I20260923 15:01:59.407822 20039 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20260923 15:01:59.408854 20054 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:01:59.409106 20039 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:01:59.409215 20039 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/minicluster-data/ts-2/wal
uuid: "430f9682c1fe49f7b9fc238400127887"
format_stamp: "Formatted at 2026-09-23 15:01:59 on dist-test-slave-cnb0"
server_key: "938d7bf162a733df869127699b1592af"
server_key_iv: "e29dabf2097eefbf2db97bbbe59360a0"
server_key_version: "encryptionkey@0"
I20260923 15:01:59.409319 20039 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:01:59.427726 20039 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:01:59.430315 20039 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:01:59.430496 20039 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:01:59.430766 20039 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 15:01:59.431183 20039 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 15:01:59.431247 20039 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 15:01:59.431281 20039 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 15:01:59.431310 20039 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 15:01:59.437659 20039 rpc_server.cc:307] RPC server started. Bound to: 127.13.75.131:35617
I20260923 15:01:59.437704 20167 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.75.131:35617 every 8 connection(s)
I20260923 15:01:59.438074 20039 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/minicluster-data/ts-2/data/info.pb
I20260923 15:01:59.438722 13614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKC4soy/build/release/bin/kudu as pid 20039
I20260923 15:01:59.438836 13614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/minicluster-data/ts-2/wal/instance
I20260923 15:01:59.439055 13614 external_mini_cluster.cc:1428] Setting key b9a751db488d19f5acbb0d43b13fb885
Sep 23 15:01:59 dist-test-slave-cnb0 krb5kdc[19664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175719, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.75.131@KRBTEST.COM for oryx/127.13.75.190@KRBTEST.COM
I20260923 15:01:59.449514 20170 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:59.439655 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.131:40787 (local address 127.13.75.190:35255)
0923 15:01:59.439771 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0923 15:01:59.439774 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:01:59.440090 (+ 316us) server_negotiation.cc:407] Connection header received
0923 15:01:59.440649 (+ 559us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:01:59.440650 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:01:59.440663 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:01:59.440726 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:01:59.441791 (+ 1065us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:59.442304 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:59.443413 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:01:59.443631 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:01:59.443723 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:01:59.446188 (+ 2465us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:01:59.446190 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:01:59.446190 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:01:59.446198 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:01:59.447843 (+ 1645us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:01:59.448365 (+ 522us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:01:59.448366 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:01:59.448366 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:01:59.448442 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:01:59.448753 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:01:59.448754 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:01:59.448755 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:01:59.448885 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:01:59.448951 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0923 15:01:59.449293 (+ 342us) server_negotiation.cc:299] Negotiation successful
0923 15:01:59.449438 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":90,"thread_start_us":62,"threads_started":1}
I20260923 15:01:59.449582 20171 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:01:59.439662 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:35255 (local address 127.13.75.131:40787)
0923 15:01:59.439851 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0923 15:01:59.439928 (+ 77us) client_negotiation.cc:174] Beginning negotiation
0923 15:01:59.440522 (+ 594us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:01:59.440805 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:01:59.440809 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:01:59.441488 (+ 679us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:01:59.441695 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:01:59.441700 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:01:59.442402 (+ 702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:01:59.442403 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:01:59.443312 (+ 909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:01:59.443315 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:01:59.443733 (+ 418us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:01:59.443734 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:01:59.443867 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:01:59.444497 (+ 630us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:01:59.444507 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:01:59.446070 (+ 1563us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:01:59.447990 (+ 1920us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:01:59.448000 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:01:59.448002 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:01:59.448270 (+ 268us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:01:59.448535 (+ 265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:01:59.448536 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:01:59.448536 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:01:59.448658 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:01:59.448977 (+ 319us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:01:59.448982 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:01:59.449129 (+ 147us) client_negotiation.cc:769] Sending connection context
0923 15:01:59.449234 (+ 105us) client_negotiation.cc:240] Negotiation successful
0923 15:01:59.449377 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":105,"threads_started":1}
I20260923 15:01:59.450328 20168 heartbeater.cc:347] Connected to a master server at 127.13.75.190:35255
I20260923 15:01:59.450448 20168 heartbeater.cc:464] Registering TS with master...
I20260923 15:01:59.450676 20168 heartbeater.cc:511] Master 127.13.75.190:35255 requested a full tablet report, sending...
I20260923 15:01:59.451381 19709 ts_manager.cc:194] Registered new tserver with Master: 430f9682c1fe49f7b9fc238400127887 (127.13.75.131:35617)
I20260923 15:01:59.451702 13614 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260923 15:01:59.451912 19709 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.13.75.131@KRBTEST.COM'} at 127.13.75.131:40787
WARNING: no policy specified for oryx/127.13.75.189@KRBTEST.COM; defaulting to no policy
I20260923 15:01:59.536211 19900 heartbeater.cc:503] Master 127.13.75.190:35255 was elected leader, sending a full tablet report...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.13.75.189@KRBTEST.COM" created.
I20260923 15:01:59.697053 20034 heartbeater.cc:503] Master 127.13.75.190:35255 was elected leader, sending a full tablet report...
I20260923 15:02:00.452965 20168 heartbeater.cc:503] Master 127.13.75.190:35255 was elected leader, sending a full tablet report...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.13.75.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/krb5kdc/oryx_127.13.75.189.keytab.
Entry for principal oryx/127.13.75.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/krb5kdc/oryx_127.13.75.189.keytab.
W20260923 15:02:00.503526 13614 mini_kdc.cc:262] Time spent creating service keytab for oryx/127.13.75.189: real 1.051s user 0.002s sys 0.001s
I20260923 15:02:00.504014 13614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKC4soy/build/release/bin/kudu
/tmp/dist-test-taskKC4soy/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.75.189:44251
--webserver_interface=127.13.75.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/krb5kdc/oryx_127.13.75.189.keytab
--principal=oryx/127.13.75.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.75.148:36053
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.75.190:35255,127.13.75.189:44251
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:02:00.590420 20174 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:02:00.590683 20174 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:02:00.590719 20174 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:02:00.592024 20174 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 15:02:00.592082 20174 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:02:00.592097 20174 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:02:00.592110 20174 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 15:02:00.592123 20174 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 15:02:00.593907 20174 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.75.148:36053
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.13.75.190:35255,127.13.75.189:44251
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/krb5kdc/oryx_127.13.75.189.keytab
--principal=oryx/127.13.75.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.75.189:44251
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.13.75.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20174
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
W20260923 15:02:00.594046 20174 master_options.cc:55] Only 2 masters are specified by master_addresses_flag ('127.13.75.190:35255,127.13.75.189:44251'), but minimum of 3 are required to tolerate failures of any one master. It is recommended to use at least 3 masters.
I20260923 15:02:00.594250 20174 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:02:00.594573 20174 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:02:00.599920 20182 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:02:00.599973 20179 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:02:00.599891 20180 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:02:00.600245 20174 server_base.cc:1034] running on GCE node
I20260923 15:02:00.600519 20174 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:02:00.600878 20174 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:02:00.602065 20174 hybrid_clock.cc:648] HybridClock initialized: now 1790175720602040 us; error 39 us; skew 500 ppm
Sep 23 15:02:00 dist-test-slave-cnb0 krb5kdc[19664](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175720, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.75.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:02:00.604969 20174 init.cc:377] Logged in from keytab as oryx/127.13.75.189@KRBTEST.COM (short username oryx)
I20260923 15:02:00.605387 20174 webserver.cc:460] Webserver started at http://127.13.75.189:44559/ using document root <none> and password file <none>
I20260923 15:02:00.605649 20174 fs_manager.cc:359] Metadata directory not provided
I20260923 15:02:00.605698 20174 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:02:00.605792 20174 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:02:00.606868 20174 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/minicluster-data/master-1/data/instance:
uuid: "b8c37de502e542aab23126e30c0e3176"
format_stamp: "Formatted at 2026-09-23 15:02:00 on dist-test-slave-cnb0"
server_key: "2ce2d4e68a2d31a4ab584de4c7d39ef0"
server_key_iv: "8627e1329cde3d131e688d90fff58c11"
server_key_version: "encryptionkey@0"
I20260923 15:02:00.607272 20174 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/minicluster-data/master-1/wal/instance:
uuid: "b8c37de502e542aab23126e30c0e3176"
format_stamp: "Formatted at 2026-09-23 15:02:00 on dist-test-slave-cnb0"
server_key: "2ce2d4e68a2d31a4ab584de4c7d39ef0"
server_key_iv: "8627e1329cde3d131e688d90fff58c11"
server_key_version: "encryptionkey@0"
I20260923 15:02:00.608656 20174 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260923 15:02:00.609555 20189 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:02:00.609750 20174 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260923 15:02:00.609865 20174 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/minicluster-data/master-1/data,/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/minicluster-data/master-1/wal
uuid: "b8c37de502e542aab23126e30c0e3176"
format_stamp: "Formatted at 2026-09-23 15:02:00 on dist-test-slave-cnb0"
server_key: "2ce2d4e68a2d31a4ab584de4c7d39ef0"
server_key_iv: "8627e1329cde3d131e688d90fff58c11"
server_key_version: "encryptionkey@0"
I20260923 15:02:00.609944 20174 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:02:00.643218 20174 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:02:00.645802 20174 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:02:00.646005 20174 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:02:00.650027 20174 rpc_server.cc:307] RPC server started. Bound to: 127.13.75.189:44251
I20260923 15:02:00.650115 20241 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.75.189:44251 every 8 connection(s)
I20260923 15:02:00.650535 20174 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/minicluster-data/master-1/data/info.pb
I20260923 15:02:00.651772 20242 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:02:00.653587 20242 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.75.190" port: 35255 } has no permanent_uuid. Determining permanent_uuid...
I20260923 15:02:00.660441 13614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKC4soy/build/release/bin/kudu as pid 20174
Sep 23 15:02:00 dist-test-slave-cnb0 krb5kdc[19664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175720, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.75.189@KRBTEST.COM for oryx/127.13.75.190@KRBTEST.COM
I20260923 15:02:00.665023 20245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:00.654525 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50772 (local address 127.13.75.190:35255)
0923 15:02:00.654669 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0923 15:02:00.654673 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 15:02:00.654742 (+ 69us) server_negotiation.cc:407] Connection header received
0923 15:02:00.655346 (+ 604us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:02:00.655347 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:02:00.655472 (+ 125us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:02:00.655557 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:02:00.656319 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:00.656955 (+ 636us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:00.658205 (+ 1250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:00.658464 (+ 259us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:00.658537 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:00.661258 (+ 2721us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:02:00.661260 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:02:00.661260 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:02:00.661269 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:02:00.663234 (+ 1965us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:02:00.663846 (+ 612us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:02:00.663847 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:02:00.663848 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:02:00.663908 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:02:00.664232 (+ 324us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:02:00.664233 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:02:00.664233 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:02:00.664378 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:02:00.664433 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0923 15:02:00.664818 (+ 385us) server_negotiation.cc:299] Negotiation successful
0923 15:02:00.664909 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":92,"threads_started":1}
I20260923 15:02:00.665162 20244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:00.654382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:35255 (local address 127.0.0.1:50772)
0923 15:02:00.654591 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0923 15:02:00.654642 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0923 15:02:00.655172 (+ 530us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:02:00.655669 (+ 497us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:02:00.655674 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:02:00.655973 (+ 299us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:02:00.656204 (+ 231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:00.656209 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:00.657057 (+ 848us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:00.657058 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:00.657938 (+ 880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:00.657941 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:00.658566 (+ 625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:00.658567 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:00.658708 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:00.659328 (+ 620us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:02:00.659339 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:02:00.661153 (+ 1814us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:02:00.663348 (+ 2195us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:02:00.663355 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:02:00.663357 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:02:00.663754 (+ 397us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:02:00.663990 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:02:00.663991 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:02:00.663991 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:02:00.664155 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:02:00.664466 (+ 311us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:02:00.664472 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:02:00.664678 (+ 206us) client_negotiation.cc:769] Sending connection context
0923 15:02:00.664776 (+ 98us) client_negotiation.cc:240] Negotiation successful
0923 15:02:00.664928 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":121,"threads_started":1}
I20260923 15:02:00.666070 20242 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.75.189" port: 44251 } has no permanent_uuid. Determining permanent_uuid...
Sep 23 15:02:00 dist-test-slave-cnb0 krb5kdc[19664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175718, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.13.75.189@KRBTEST.COM
Sep 23 15:02:00 dist-test-slave-cnb0 krb5kdc[19664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175720, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.75.189@KRBTEST.COM for oryx/127.13.75.189@KRBTEST.COM
I20260923 15:02:00.671643 20247 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:00.660996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39098 (local address 127.13.75.189:44251)
0923 15:02:00.661222 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0923 15:02:00.661228 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 15:02:00.661249 (+ 21us) server_negotiation.cc:407] Connection header received
0923 15:02:00.661323 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:02:00.661326 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:02:00.661342 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:02:00.661427 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:02:00.662319 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:00.663473 (+ 1154us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:00.664608 (+ 1135us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:00.665004 (+ 396us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:00.665078 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:00.667984 (+ 2906us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:02:00.667991 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:02:00.667992 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:02:00.667999 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:02:00.670098 (+ 2099us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:02:00.670528 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:02:00.670531 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:02:00.670532 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:02:00.670591 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:02:00.670826 (+ 235us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:02:00.670826 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:02:00.670827 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:02:00.671076 (+ 249us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:02:00.671261 (+ 185us) server_negotiation.cc:1035] Waiting for connection context
0923 15:02:00.671359 (+ 98us) server_negotiation.cc:299] Negotiation successful
0923 15:02:00.671460 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":190,"thread_start_us":156,"threads_started":1}
I20260923 15:02:00.674463 20244 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:00.666362 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.189:44251 (local address 127.0.0.1:39114)
0923 15:02:00.666629 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0923 15:02:00.666642 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0923 15:02:00.666699 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:02:00.666857 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:02:00.666858 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:02:00.667059 (+ 201us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:02:00.667110 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:00.667111 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:00.668007 (+ 896us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:00.668008 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:00.668669 (+ 661us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:00.668672 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:00.669218 (+ 546us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:00.669219 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:00.669297 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:00.669781 (+ 484us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:02:00.669786 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:02:00.671464 (+ 1678us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:02:00.673276 (+ 1812us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:02:00.673277 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:02:00.673277 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:02:00.673509 (+ 232us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:02:00.673745 (+ 236us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:02:00.673746 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:02:00.673747 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:02:00.673806 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:02:00.674083 (+ 277us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:02:00.674084 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:02:00.674124 (+ 40us) client_negotiation.cc:769] Sending connection context
0923 15:02:00.674200 (+ 76us) client_negotiation.cc:240] Negotiation successful
0923 15:02:00.674329 (+ 129us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":240}
I20260923 15:02:00.674491 20248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:00.666475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39114 (local address 127.13.75.189:44251)
0923 15:02:00.666633 (+ 158us) server_negotiation.cc:206] Beginning negotiation
0923 15:02:00.666635 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:02:00.666681 (+ 46us) server_negotiation.cc:407] Connection header received
0923 15:02:00.666759 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:02:00.666760 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:02:00.666771 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:02:00.666843 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:02:00.667195 (+ 352us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:00.667881 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:00.668766 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:00.669086 (+ 320us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:00.669219 (+ 133us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:00.671632 (+ 2413us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:02:00.671634 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:02:00.671635 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:02:00.671642 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:02:00.673179 (+ 1537us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:02:00.673606 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:02:00.673607 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:02:00.673608 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:02:00.673674 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:02:00.673893 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:02:00.673894 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:02:00.673894 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:02:00.674010 (+ 116us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:02:00.674093 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0923 15:02:00.674226 (+ 133us) server_negotiation.cc:299] Negotiation successful
0923 15:02:00.674361 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":67,"server-negotiator.queue_time_us":135,"thread_start_us":106,"threads_started":1}
I20260923 15:02:00.675997 20242 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b8c37de502e542aab23126e30c0e3176: Bootstrap starting.
I20260923 15:02:00.676645 20242 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b8c37de502e542aab23126e30c0e3176: Neither blocks nor log segments found. Creating new log.
I20260923 15:02:00.676934 20242 log.cc:824] T 00000000000000000000000000000000 P b8c37de502e542aab23126e30c0e3176: Log is configured to *not* fsync() on all Append() calls
I20260923 15:02:00.677809 20242 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b8c37de502e542aab23126e30c0e3176: No bootstrap required, opened a new log
I20260923 15:02:00.679384 20242 raft_consensus.cc:348] T 00000000000000000000000000000000 P b8c37de502e542aab23126e30c0e3176 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "176378cdc0584ce69eca703063507558" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 35255 } } peers { permanent_uuid: "b8c37de502e542aab23126e30c0e3176" member_type: VOTER last_known_addr { host: "127.13.75.189" port: 44251 } }
I20260923 15:02:00.679550 20242 raft_consensus.cc:374] T 00000000000000000000000000000000 P b8c37de502e542aab23126e30c0e3176 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:02:00.679600 20242 raft_consensus.cc:729] T 00000000000000000000000000000000 P b8c37de502e542aab23126e30c0e3176 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b8c37de502e542aab23126e30c0e3176, State: Initialized, Role: FOLLOWER
I20260923 15:02:00.679747 20242 consensus_queue.cc:260] T 00000000000000000000000000000000 P b8c37de502e542aab23126e30c0e3176 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "176378cdc0584ce69eca703063507558" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 35255 } } peers { permanent_uuid: "b8c37de502e542aab23126e30c0e3176" member_type: VOTER last_known_addr { host: "127.13.75.189" port: 44251 } }
I20260923 15:02:00.680043 20250 sys_catalog.cc:455] T 00000000000000000000000000000000 P b8c37de502e542aab23126e30c0e3176 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "176378cdc0584ce69eca703063507558" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 35255 } } peers { permanent_uuid: "b8c37de502e542aab23126e30c0e3176" member_type: VOTER last_known_addr { host: "127.13.75.189" port: 44251 } } }
I20260923 15:02:00.680128 20242 sys_catalog.cc:564] T 00000000000000000000000000000000 P b8c37de502e542aab23126e30c0e3176 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 15:02:00.680135 20250 sys_catalog.cc:458] T 00000000000000000000000000000000 P b8c37de502e542aab23126e30c0e3176 [sys.catalog]: This master's current role is: FOLLOWER
W20260923 15:02:00.681967 20261 catalog_manager.cc:1558] T 00000000000000000000000000000000 P b8c37de502e542aab23126e30c0e3176: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260923 15:02:00.682055 20261 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260923 15:02:00.683661 20250 raft_consensus.cc:482] T 00000000000000000000000000000000 P b8c37de502e542aab23126e30c0e3176 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 15:02:00.683755 20250 raft_consensus.cc:504] T 00000000000000000000000000000000 P b8c37de502e542aab23126e30c0e3176 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "176378cdc0584ce69eca703063507558" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 35255 } } peers { permanent_uuid: "b8c37de502e542aab23126e30c0e3176" member_type: VOTER last_known_addr { host: "127.13.75.189" port: 44251 } }
I20260923 15:02:00.683980 20250 leader_election.cc:288] T 00000000000000000000000000000000 P b8c37de502e542aab23126e30c0e3176 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 176378cdc0584ce69eca703063507558 (127.13.75.190:35255)
I20260923 15:02:00.684340 19719 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b8c37de502e542aab23126e30c0e3176" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "176378cdc0584ce69eca703063507558" is_pre_election: true
I20260923 15:02:00.684568 20174 master_runner.cc:428] Detected that this master b8c37de502e542aab23126e30c0e3176 is joining an existing cluster
I20260923 15:02:00.684613 20174 master_runner.cc:432] Initiating AddMaster RPC to add 127.13.75.189:44251
I20260923 15:02:00.684727 20191 leader_election.cc:302] T 00000000000000000000000000000000 P b8c37de502e542aab23126e30c0e3176 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 2 responses out of 2 voters: 1 yes votes; 1 no votes. yes voters: b8c37de502e542aab23126e30c0e3176; no voters: 176378cdc0584ce69eca703063507558
I20260923 15:02:00.684911 20250 raft_consensus.cc:2726] T 00000000000000000000000000000000 P b8c37de502e542aab23126e30c0e3176 [term 0 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260923 15:02:00.693715 20268 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:00.685724 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:35255 (local address 127.0.0.1:50784)
0923 15:02:00.685907 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0923 15:02:00.685918 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 15:02:00.685973 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:02:00.686395 (+ 422us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:02:00.686396 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:02:00.686676 (+ 280us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:02:00.686769 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:00.686770 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:00.687748 (+ 978us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:00.687749 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:00.688640 (+ 891us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:00.688642 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:00.689134 (+ 492us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:00.689136 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:00.689202 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:00.689789 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:02:00.689794 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:02:00.690316 (+ 522us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:02:00.692390 (+ 2074us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:02:00.692399 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:02:00.692399 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:02:00.692717 (+ 318us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:02:00.692979 (+ 262us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:02:00.692980 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:02:00.692980 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:02:00.693049 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:02:00.693347 (+ 298us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:02:00.693348 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:02:00.693413 (+ 65us) client_negotiation.cc:769] Sending connection context
0923 15:02:00.693483 (+ 70us) client_negotiation.cc:240] Negotiation successful
0923 15:02:00.693603 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":155,"mutex_wait_us":2,"thread_start_us":127,"threads_started":1}
I20260923 15:02:00.694029 20248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:00.685762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39120 (local address 127.13.75.189:44251)
0923 15:02:00.686116 (+ 354us) server_negotiation.cc:206] Beginning negotiation
0923 15:02:00.686119 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:02:00.686132 (+ 13us) server_negotiation.cc:407] Connection header received
0923 15:02:00.686157 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:02:00.686158 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:02:00.686169 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:02:00.686236 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:02:00.686694 (+ 458us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:00.687380 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:00.688544 (+ 1164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:00.688867 (+ 323us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:00.688944 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:00.690552 (+ 1608us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:02:00.690553 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:02:00.690553 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:02:00.690560 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:02:00.692724 (+ 2164us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:02:00.693084 (+ 360us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:02:00.693085 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:02:00.693085 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:02:00.693129 (+ 44us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:02:00.693307 (+ 178us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:02:00.693308 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:02:00.693309 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:02:00.693486 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:02:00.693556 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0923 15:02:00.693728 (+ 172us) server_negotiation.cc:299] Negotiation successful
0923 15:02:00.693944 (+ 216us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":20,"server-negotiator.queue_time_us":331}
I20260923 15:02:00.694114 20245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:00.686109 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50784 (local address 127.13.75.190:35255)
0923 15:02:00.686284 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0923 15:02:00.686286 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:02:00.686292 (+ 6us) server_negotiation.cc:407] Connection header received
0923 15:02:00.686311 (+ 19us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:02:00.686311 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:02:00.686319 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:02:00.686390 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:02:00.686858 (+ 468us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:00.687656 (+ 798us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:00.688727 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:00.689048 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:00.689126 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:00.690420 (+ 1294us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:02:00.690422 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:02:00.690423 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:02:00.690432 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:02:00.692272 (+ 1840us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:02:00.692815 (+ 543us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:02:00.692816 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:02:00.692817 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:02:00.692889 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:02:00.693120 (+ 231us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:02:00.693121 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:02:00.693121 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:02:00.693284 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:02:00.693363 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0923 15:02:00.693530 (+ 167us) server_negotiation.cc:299] Negotiation successful
0923 15:02:00.693640 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":154}
I20260923 15:02:00.694352 20267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:00.685725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.189:44251 (local address 127.0.0.1:39120)
0923 15:02:00.686060 (+ 335us) negotiation.cc:107] Waiting for socket to connect
0923 15:02:00.686068 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0923 15:02:00.686100 (+ 32us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:02:00.686262 (+ 162us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:02:00.686263 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:02:00.686513 (+ 250us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:02:00.686607 (+ 94us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:00.686609 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:00.687489 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:00.687490 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:00.688447 (+ 957us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:00.688449 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:00.688973 (+ 524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:00.688974 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:00.689048 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:00.689604 (+ 556us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:02:00.689610 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:02:00.690458 (+ 848us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:02:00.692813 (+ 2355us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:02:00.692813 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:02:00.692814 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:02:00.693008 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:02:00.693190 (+ 182us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:02:00.693190 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:02:00.693191 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:02:00.693236 (+ 45us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:02:00.693580 (+ 344us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:02:00.693580 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:02:00.693623 (+ 43us) client_negotiation.cc:769] Sending connection context
0923 15:02:00.694144 (+ 521us) client_negotiation.cc:240] Negotiation successful
0923 15:02:00.694255 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":322,"mutex_wait_us":2,"thread_start_us":104,"threads_started":1}
Sep 23 15:02:00 dist-test-slave-cnb0 krb5kdc[19664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175718, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.75.190@KRBTEST.COM for oryx/127.13.75.189@KRBTEST.COM
I20260923 15:02:00.705184 20248 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:00.696765 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39126 (local address 127.13.75.189:44251)
0923 15:02:00.696814 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0923 15:02:00.696815 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 15:02:00.696976 (+ 161us) server_negotiation.cc:407] Connection header received
0923 15:02:00.697060 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:02:00.697061 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:02:00.697070 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:02:00.697117 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:02:00.697663 (+ 546us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:00.698196 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:00.699385 (+ 1189us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:00.699584 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:00.699673 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:00.702229 (+ 2556us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:02:00.702231 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:02:00.702232 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:02:00.702240 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:02:00.703804 (+ 1564us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:02:00.704271 (+ 467us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:02:00.704272 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:02:00.704272 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:02:00.704341 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:02:00.704557 (+ 216us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:02:00.704558 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:02:00.704558 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:02:00.704699 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:02:00.704781 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0923 15:02:00.704988 (+ 207us) server_negotiation.cc:299] Negotiation successful
0923 15:02:00.705080 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260923 15:02:00.705195 20269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:00.696608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.189:44251 (local address 127.0.0.1:39126)
0923 15:02:00.696853 (+ 245us) negotiation.cc:107] Waiting for socket to connect
0923 15:02:00.696901 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0923 15:02:00.696981 (+ 80us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:02:00.697131 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:02:00.697134 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:02:00.697478 (+ 344us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:02:00.697553 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:00.697555 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:00.698298 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:00.698299 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:00.699295 (+ 996us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:00.699298 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:00.699690 (+ 392us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:00.699691 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:00.699783 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:00.700335 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:02:00.700341 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:02:00.702119 (+ 1778us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:02:00.703937 (+ 1818us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:02:00.703948 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:02:00.703950 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:02:00.704185 (+ 235us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:02:00.704417 (+ 232us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:02:00.704418 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:02:00.704419 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:02:00.704491 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:02:00.704772 (+ 281us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:02:00.704782 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:02:00.704843 (+ 61us) client_negotiation.cc:769] Sending connection context
0923 15:02:00.704958 (+ 115us) client_negotiation.cc:240] Negotiation successful
0923 15:02:00.705097 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":215,"thread_start_us":178,"threads_started":1}
I20260923 15:02:00.706032 19709 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.13.75.189:44251: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "b8c37de502e542aab23126e30c0e3176"
member_type: NON_VOTER
last_known_addr {
host: "127.13.75.189"
port: 44251
}
attrs {
promote: true
}
}
dest_uuid: "176378cdc0584ce69eca703063507558"
cas_config_opid_index: -1
I20260923 15:02:00.706405 19709 consensus_queue.cc:237] T 00000000000000000000000000000000 P 176378cdc0584ce69eca703063507558 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "176378cdc0584ce69eca703063507558" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 35255 } } peers { permanent_uuid: "b8c37de502e542aab23126e30c0e3176" member_type: NON_VOTER last_known_addr { host: "127.13.75.189" port: 44251 } attrs { promote: true } }
I20260923 15:02:00.707065 20271 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 176378cdc0584ce69eca703063507558 [term 1 LEADER]: Committing config change with OpId 1.5: config changed from index -1 to 5, NON_VOTER b8c37de502e542aab23126e30c0e3176 (127.13.75.189) added. New config: { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "176378cdc0584ce69eca703063507558" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 35255 } } peers { permanent_uuid: "b8c37de502e542aab23126e30c0e3176" member_type: NON_VOTER last_known_addr { host: "127.13.75.189" port: 44251 } attrs { promote: true } } }
I20260923 15:02:00.707444 20216 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b8c37de502e542aab23126e30c0e3176 [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:02:00.708237 20271 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.13.75.189:44251
I20260923 15:02:00.708329 20271 sys_catalog.cc:455] T 00000000000000000000000000000000 P 176378cdc0584ce69eca703063507558 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "176378cdc0584ce69eca703063507558" committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "176378cdc0584ce69eca703063507558" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 35255 } } peers { permanent_uuid: "b8c37de502e542aab23126e30c0e3176" member_type: NON_VOTER last_known_addr { host: "127.13.75.189" port: 44251 } attrs { promote: true } } }
I20260923 15:02:00.708431 20271 sys_catalog.cc:458] T 00000000000000000000000000000000 P 176378cdc0584ce69eca703063507558 [sys.catalog]: This master's current role is: LEADER
I20260923 15:02:00.708530 20216 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b8c37de502e542aab23126e30c0e3176 [term 1 FOLLOWER]: Refusing update from remote peer 176378cdc0584ce69eca703063507558: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 5. (index mismatch)
I20260923 15:02:00.708855 20174 master.cc:537] Master@127.13.75.189:44251 shutting down...
I20260923 15:02:00.709028 20271 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 176378cdc0584ce69eca703063507558 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8c37de502e542aab23126e30c0e3176" member_type: NON_VOTER last_known_addr { host: "127.13.75.189" port: 44251 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 5, Last known committed idx: 0, Time since last communication: 0.000s
W20260923 15:02:00.709834 19695 proxy.cc:239] Call had error, refreshing address and retrying: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master
W20260923 15:02:00.710281 19695 consensus_peers.cc:487] T 00000000000000000000000000000000 P 176378cdc0584ce69eca703063507558 -> Peer b8c37de502e542aab23126e30c0e3176 (127.13.75.189:44251): Couldn't send request to peer b8c37de502e542aab23126e30c0e3176. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260923 15:02:00.712149 20174 raft_consensus.cc:2229] T 00000000000000000000000000000000 P b8c37de502e542aab23126e30c0e3176 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 15:02:00.712239 20174 raft_consensus.cc:2258] T 00000000000000000000000000000000 P b8c37de502e542aab23126e30c0e3176 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 15:02:00.712285 20174 tablet_replica.cc:330] T 00000000000000000000000000000000 P b8c37de502e542aab23126e30c0e3176: stopping tablet replica
I20260923 15:02:00.716357 20174 master.cc:552] Master@127.13.75.189:44251 shutdown complete.
I20260923 15:02:00.716413 20174 master_runner.cc:305] Clearing existing system tablet
I20260923 15:02:00.716559 20174 fs_manager.cc:359] Metadata directory not provided
I20260923 15:02:00.716611 20174 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:02:00.717409 20174 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:02:00.718003 20276 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:02:00.718165 20174 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:02:00.718232 20174 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260923 15:02:00.718259 20174 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/minicluster-data/master-1/data,/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/minicluster-data/master-1/wal
uuid: "b8c37de502e542aab23126e30c0e3176"
format_stamp: "Formatted at 2026-09-23 15:02:00 on dist-test-slave-cnb0"
server_key: "2ce2d4e68a2d31a4ab584de4c7d39ef0"
server_key_iv: "8627e1329cde3d131e688d90fff58c11"
server_key_version: "encryptionkey@0"
I20260923 15:02:00.719090 20174 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P b8c37de502e542aab23126e30c0e3176: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 15:02:00.720737 20174 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P b8c37de502e542aab23126e30c0e3176: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260923 15:02:00.720817 20174 log.cc:1196] T 00000000000000000000000000000000 P b8c37de502e542aab23126e30c0e3176: Deleting WAL directory at /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/minicluster-data/master-1/wal/wals/00000000000000000000000000000000
I20260923 15:02:00.721019 20174 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P b8c37de502e542aab23126e30c0e3176: Deleting consensus metadata
I20260923 15:02:00.721159 20174 master_runner.cc:315] Copying system tablet from 127.13.75.190:35255
I20260923 15:02:00.721817 20174 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P b8c37de502e542aab23126e30c0e3176: tablet copy: Beginning tablet copy session from remote peer at address 127.13.75.190:35255
I20260923 15:02:00.728874 20282 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:00.722244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:35255 (local address 127.0.0.1:50788)
0923 15:02:00.722451 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0923 15:02:00.722461 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 15:02:00.722523 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:02:00.722680 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:02:00.722681 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:02:00.722845 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:02:00.722960 (+ 115us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:00.722964 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:00.724003 (+ 1039us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:00.724004 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:00.724627 (+ 623us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:00.724629 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:00.725139 (+ 510us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:00.725140 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:00.725202 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:00.725720 (+ 518us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:02:00.725724 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:02:00.726060 (+ 336us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:02:00.727902 (+ 1842us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:02:00.727904 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:02:00.727904 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:02:00.728085 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:02:00.728292 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:02:00.728293 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:02:00.728293 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:02:00.728344 (+ 51us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:02:00.728610 (+ 266us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:02:00.728611 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:02:00.728650 (+ 39us) client_negotiation.cc:769] Sending connection context
0923 15:02:00.728714 (+ 64us) client_negotiation.cc:240] Negotiation successful
0923 15:02:00.728799 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":175,"thread_start_us":85,"threads_started":1}
I20260923 15:02:00.728945 20245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:00.722307 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50788 (local address 127.13.75.190:35255)
0923 15:02:00.722389 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0923 15:02:00.722392 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:02:00.722533 (+ 141us) server_negotiation.cc:407] Connection header received
0923 15:02:00.722572 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:02:00.722573 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:02:00.722585 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:02:00.722669 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:02:00.723064 (+ 395us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:00.723890 (+ 826us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:00.724719 (+ 829us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:00.725021 (+ 302us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:00.725095 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:00.726152 (+ 1057us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:02:00.726154 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:02:00.726154 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:02:00.726161 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:02:00.727815 (+ 1654us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:02:00.728161 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:02:00.728162 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:02:00.728163 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:02:00.728226 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:02:00.728402 (+ 176us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:02:00.728403 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:02:00.728403 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:02:00.728547 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:02:00.728616 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0923 15:02:00.728759 (+ 143us) server_negotiation.cc:299] Negotiation successful
0923 15:02:00.728846 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260923 15:02:00.729588 19729 tablet_copy_service.cc:140] P 176378cdc0584ce69eca703063507558: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer b8c37de502e542aab23126e30c0e3176 ({username='oryx', principal='oryx/127.13.75.189@KRBTEST.COM'} at 127.0.0.1:50788)
I20260923 15:02:00.729684 19729 tablet_copy_service.cc:161] P 176378cdc0584ce69eca703063507558: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer b8c37de502e542aab23126e30c0e3176 at {username='oryx', principal='oryx/127.13.75.189@KRBTEST.COM'} at 127.0.0.1:50788: session id = b8c37de502e542aab23126e30c0e3176-00000000000000000000000000000000
I20260923 15:02:00.730350 19729 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 176378cdc0584ce69eca703063507558: Tablet Copy: opened 0 blocks and 1 log segments
I20260923 15:02:00.731042 20174 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260923 15:02:00.732432 20174 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P b8c37de502e542aab23126e30c0e3176: tablet copy: Starting download of 0 data blocks...
I20260923 15:02:00.732609 20174 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P b8c37de502e542aab23126e30c0e3176: tablet copy: Starting download of 1 WAL segments...
I20260923 15:02:00.733671 20174 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P b8c37de502e542aab23126e30c0e3176: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260923 15:02:00.734777 19729 tablet_copy_service.cc:342] P 176378cdc0584ce69eca703063507558: Request end of tablet copy session b8c37de502e542aab23126e30c0e3176-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.13.75.189@KRBTEST.COM'} at 127.0.0.1:50788
I20260923 15:02:00.734879 19729 tablet_copy_service.cc:433] P 176378cdc0584ce69eca703063507558: ending tablet copy session b8c37de502e542aab23126e30c0e3176-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer b8c37de502e542aab23126e30c0e3176
I20260923 15:02:00.738039 20174 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:02:00.739593 20284 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:02:00.739698 20283 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:02:00.744283 20286 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:02:00.744313 20174 server_base.cc:1034] running on GCE node
I20260923 15:02:00.744540 20174 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:02:00.744700 20174 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:02:00.745832 20174 hybrid_clock.cc:648] HybridClock initialized: now 1790175720745820 us; error 30 us; skew 500 ppm
Sep 23 15:02:00 dist-test-slave-cnb0 krb5kdc[19664](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175720, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.75.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:02:00.747403 20174 init.cc:377] Logged in from keytab as oryx/127.13.75.189@KRBTEST.COM (short username oryx)
I20260923 15:02:00.747733 20174 webserver.cc:460] Webserver started at http://127.13.75.189:37437/ using document root <none> and password file <none>
I20260923 15:02:00.747880 20174 fs_manager.cc:359] Metadata directory not provided
I20260923 15:02:00.747924 20174 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:02:00.748600 20174 fs_manager.cc:709] Time spent opening directory manager: real 0.000s user 0.001s sys 0.000s
I20260923 15:02:00.749188 20293 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:02:00.749328 20174 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 15:02:00.749384 20174 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/minicluster-data/master-1/data,/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/minicluster-data/master-1/wal
uuid: "b8c37de502e542aab23126e30c0e3176"
format_stamp: "Formatted at 2026-09-23 15:02:00 on dist-test-slave-cnb0"
server_key: "2ce2d4e68a2d31a4ab584de4c7d39ef0"
server_key_iv: "8627e1329cde3d131e688d90fff58c11"
server_key_version: "encryptionkey@0"
I20260923 15:02:00.749447 20174 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:02:00.793135 20174 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:02:00.795190 20174 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:02:00.799069 20174 rpc_server.cc:307] RPC server started. Bound to: 127.13.75.189:44251
I20260923 15:02:00.799237 20345 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.75.189:44251 every 8 connection(s)
I20260923 15:02:00.799494 20174 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/minicluster-data/master-1/data/info.pb
I20260923 15:02:00.800591 20346 sys_catalog.cc:263] Verifying existing consensus state
I20260923 15:02:00.801146 20346 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b8c37de502e542aab23126e30c0e3176: Bootstrap starting.
I20260923 15:02:00.805357 20346 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b8c37de502e542aab23126e30c0e3176: Bootstrap replayed 1/1 log segments. Stats: ops{read=5 overwritten=0 applied=5 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260923 15:02:00.805675 20346 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b8c37de502e542aab23126e30c0e3176: Bootstrap complete.
I20260923 15:02:00.805975 20346 raft_consensus.cc:348] T 00000000000000000000000000000000 P b8c37de502e542aab23126e30c0e3176 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "176378cdc0584ce69eca703063507558" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 35255 } } peers { permanent_uuid: "b8c37de502e542aab23126e30c0e3176" member_type: NON_VOTER last_known_addr { host: "127.13.75.189" port: 44251 } attrs { promote: true } }
I20260923 15:02:00.806089 20346 raft_consensus.cc:729] T 00000000000000000000000000000000 P b8c37de502e542aab23126e30c0e3176 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: b8c37de502e542aab23126e30c0e3176, State: Initialized, Role: LEARNER
I20260923 15:02:00.806140 20346 consensus_queue.cc:260] T 00000000000000000000000000000000 P b8c37de502e542aab23126e30c0e3176 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 5, Last appended: 1.5, Last appended by leader: 5, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "176378cdc0584ce69eca703063507558" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 35255 } } peers { permanent_uuid: "b8c37de502e542aab23126e30c0e3176" member_type: NON_VOTER last_known_addr { host: "127.13.75.189" port: 44251 } attrs { promote: true } }
I20260923 15:02:00.806340 20346 sys_catalog.cc:564] T 00000000000000000000000000000000 P b8c37de502e542aab23126e30c0e3176 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 15:02:00.806483 20349 sys_catalog.cc:455] T 00000000000000000000000000000000 P b8c37de502e542aab23126e30c0e3176 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 5 OBSOLETE_local: true peers { permanent_uuid: "176378cdc0584ce69eca703063507558" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 35255 } } peers { permanent_uuid: "b8c37de502e542aab23126e30c0e3176" member_type: NON_VOTER last_known_addr { host: "127.13.75.189" port: 44251 } attrs { promote: true } } }
I20260923 15:02:00.806623 20349 sys_catalog.cc:458] T 00000000000000000000000000000000 P b8c37de502e542aab23126e30c0e3176 [sys.catalog]: This master's current role is: LEARNER
I20260923 15:02:00.808094 20360 catalog_manager.cc:1259] Loaded cluster ID: 4b5be46274e44cd197c2c043e352da13
I20260923 15:02:00.808211 20360 catalog_manager.cc:1552] T 00000000000000000000000000000000 P b8c37de502e542aab23126e30c0e3176: loading cluster ID for follower catalog manager: success
I20260923 15:02:00.809053 20360 catalog_manager.cc:1574] T 00000000000000000000000000000000 P b8c37de502e542aab23126e30c0e3176: acquiring CA information for follower catalog manager: success
I20260923 15:02:00.809355 20360 catalog_manager.cc:1602] T 00000000000000000000000000000000 P b8c37de502e542aab23126e30c0e3176: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260923 15:02:00.833541 20361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:00.824484 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39146 (local address 127.13.75.189:44251)
0923 15:02:00.824646 (+ 162us) server_negotiation.cc:206] Beginning negotiation
0923 15:02:00.824649 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:02:00.824675 (+ 26us) server_negotiation.cc:407] Connection header received
0923 15:02:00.824704 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:02:00.824705 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:02:00.824717 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:02:00.824798 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:02:00.825433 (+ 635us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:00.826537 (+ 1104us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:00.827535 (+ 998us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:00.827872 (+ 337us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:00.827948 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:00.829321 (+ 1373us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:02:00.829323 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:02:00.829324 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:02:00.829333 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:02:00.832033 (+ 2700us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:02:00.832502 (+ 469us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:02:00.832504 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:02:00.832504 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:02:00.832626 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:02:00.832884 (+ 258us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:02:00.832885 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:02:00.832886 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:02:00.833061 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:02:00.833192 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0923 15:02:00.833283 (+ 91us) server_negotiation.cc:299] Negotiation successful
0923 15:02:00.833392 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":139,"thread_start_us":97,"threads_started":1}
WARNING: no policy specified for oryx/127.13.75.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.13.75.188@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.13.75.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/krb5kdc/oryx_127.13.75.188.keytab.
Entry for principal oryx/127.13.75.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/krb5kdc/oryx_127.13.75.188.keytab.
I20260923 15:02:00.862810 13614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKC4soy/build/release/bin/kudu
/tmp/dist-test-taskKC4soy/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.75.188:33585
--webserver_interface=127.13.75.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/krb5kdc/oryx_127.13.75.188.keytab
--principal=oryx/127.13.75.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.75.148:36053
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.75.190:35255,127.13.75.189:44251,127.13.75.188:33585
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:02:00.947232 20364 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:02:00.947461 20364 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:02:00.947486 20364 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:02:00.948827 20364 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 15:02:00.948886 20364 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:02:00.948901 20364 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:02:00.948915 20364 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 15:02:00.948926 20364 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 15:02:00.950673 20364 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.75.148:36053
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.13.75.190:35255,127.13.75.189:44251,127.13.75.188:33585
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/krb5kdc/oryx_127.13.75.188.keytab
--principal=oryx/127.13.75.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.75.188:33585
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.13.75.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20364
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:02:00.950938 20364 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:02:00.951169 20364 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:02:00.956209 20370 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:02:00.956251 20364 server_base.cc:1034] running on GCE node
W20260923 15:02:00.956231 20369 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:02:00.956202 20372 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:02:00.956765 20364 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:02:00.957168 20364 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:02:00.958359 20364 hybrid_clock.cc:648] HybridClock initialized: now 1790175720958348 us; error 55 us; skew 500 ppm
Sep 23 15:02:00 dist-test-slave-cnb0 krb5kdc[19664](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175720, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.75.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:02:00.961213 20364 init.cc:377] Logged in from keytab as oryx/127.13.75.188@KRBTEST.COM (short username oryx)
I20260923 15:02:00.961773 20364 webserver.cc:460] Webserver started at http://127.13.75.188:37775/ using document root <none> and password file <none>
I20260923 15:02:00.962010 20364 fs_manager.cc:359] Metadata directory not provided
I20260923 15:02:00.962057 20364 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:02:00.962159 20364 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:02:00.963383 20364 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/minicluster-data/master-2/data/instance:
uuid: "28aa43d448d94ccdb6d8776e93de4fc9"
format_stamp: "Formatted at 2026-09-23 15:02:00 on dist-test-slave-cnb0"
server_key: "de9a8214d89a2004e0f482bbc54b2df1"
server_key_iv: "f38f585feaa2efbcc266e0e2a47ff902"
server_key_version: "encryptionkey@0"
I20260923 15:02:00.963739 20364 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/minicluster-data/master-2/wal/instance:
uuid: "28aa43d448d94ccdb6d8776e93de4fc9"
format_stamp: "Formatted at 2026-09-23 15:02:00 on dist-test-slave-cnb0"
server_key: "de9a8214d89a2004e0f482bbc54b2df1"
server_key_iv: "f38f585feaa2efbcc266e0e2a47ff902"
server_key_version: "encryptionkey@0"
I20260923 15:02:00.965209 20364 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 15:02:00.966121 20379 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:02:00.966305 20364 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:02:00.966387 20364 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/minicluster-data/master-2/data,/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/minicluster-data/master-2/wal
uuid: "28aa43d448d94ccdb6d8776e93de4fc9"
format_stamp: "Formatted at 2026-09-23 15:02:00 on dist-test-slave-cnb0"
server_key: "de9a8214d89a2004e0f482bbc54b2df1"
server_key_iv: "f38f585feaa2efbcc266e0e2a47ff902"
server_key_version: "encryptionkey@0"
I20260923 15:02:00.966444 20364 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:02:00.985260 20364 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:02:00.987870 20364 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:02:00.987998 20364 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:02:00.992216 20364 rpc_server.cc:307] RPC server started. Bound to: 127.13.75.188:33585
I20260923 15:02:00.992256 20431 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.75.188:33585 every 8 connection(s)
I20260923 15:02:00.992635 20364 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/minicluster-data/master-2/data/info.pb
I20260923 15:02:00.993729 20432 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:02:00.995497 20432 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.75.190" port: 35255 } has no permanent_uuid. Determining permanent_uuid...
I20260923 15:02:00.998684 13614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKC4soy/build/release/bin/kudu as pid 20364
Sep 23 15:02:00 dist-test-slave-cnb0 krb5kdc[19664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175720, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.75.188@KRBTEST.COM for oryx/127.13.75.190@KRBTEST.COM
Sep 23 15:02:00 dist-test-slave-cnb0 krb5kdc[19664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175718, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.13.75.188@KRBTEST.COM
I20260923 15:02:01.009861 20245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:00.996665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50792 (local address 127.13.75.190:35255)
0923 15:02:00.996741 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0923 15:02:00.996743 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:02:00.996919 (+ 176us) server_negotiation.cc:407] Connection header received
0923 15:02:00.997642 (+ 723us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:02:00.997644 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:02:00.997659 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:02:00.997712 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:02:00.998526 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:00.999369 (+ 843us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:01.001315 (+ 1946us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:01.001520 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:01.001980 (+ 460us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:01.006188 (+ 4208us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:02:01.006190 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:02:01.006191 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:02:01.006200 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:02:01.008231 (+ 2031us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:02:01.008713 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:02:01.008715 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:02:01.008716 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:02:01.008786 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:02:01.009068 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:02:01.009069 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:02:01.009070 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:02:01.009208 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:02:01.009285 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0923 15:02:01.009653 (+ 368us) server_negotiation.cc:299] Negotiation successful
0923 15:02:01.009759 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260923 15:02:01.010082 20434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:00.996382 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:35255 (local address 127.0.0.1:50792)
0923 15:02:00.996652 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0923 15:02:00.996731 (+ 79us) client_negotiation.cc:174] Beginning negotiation
0923 15:02:00.997505 (+ 774us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:02:00.997801 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:02:00.997805 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:02:00.998126 (+ 321us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:02:00.998365 (+ 239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:00.998369 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:00.999747 (+ 1378us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:00.999757 (+ 10us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:01.001221 (+ 1464us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:01.001225 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:01.002078 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:01.002080 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:01.002209 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:01.003635 (+ 1426us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:02:01.003647 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:02:01.006064 (+ 2417us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:02:01.008329 (+ 2265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:02:01.008339 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:02:01.008342 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:02:01.008608 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:02:01.008857 (+ 249us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:02:01.008858 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:02:01.008859 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:02:01.008992 (+ 133us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:02:01.009284 (+ 292us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:02:01.009292 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:02:01.009459 (+ 167us) client_negotiation.cc:769] Sending connection context
0923 15:02:01.009633 (+ 174us) client_negotiation.cc:240] Negotiation successful
0923 15:02:01.009813 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":213,"thread_start_us":134,"threads_started":1}
I20260923 15:02:01.010927 20432 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.75.189" port: 44251 } has no permanent_uuid. Determining permanent_uuid...
I20260923 15:02:01.011101 20435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:00.999453 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40998 (local address 127.13.75.188:33585)
0923 15:02:00.999678 (+ 225us) server_negotiation.cc:206] Beginning negotiation
0923 15:02:00.999683 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 15:02:00.999981 (+ 298us) server_negotiation.cc:407] Connection header received
0923 15:02:01.000009 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:02:01.000010 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:02:01.000037 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:02:01.000110 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:02:01.001149 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:01.001919 (+ 770us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:01.002665 (+ 746us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:01.003046 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:01.003129 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:01.006457 (+ 3328us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:02:01.006621 (+ 164us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:02:01.006622 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:02:01.006632 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:02:01.009366 (+ 2734us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:02:01.009776 (+ 410us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:02:01.009779 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:02:01.009780 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:02:01.009843 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:02:01.010119 (+ 276us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:02:01.010120 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:02:01.010121 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:02:01.010374 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:02:01.010543 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0923 15:02:01.010745 (+ 202us) server_negotiation.cc:299] Negotiation successful
0923 15:02:01.010837 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":188,"thread_start_us":155,"threads_started":1}
Sep 23 15:02:01 dist-test-slave-cnb0 krb5kdc[19664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175720, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.75.188@KRBTEST.COM for oryx/127.13.75.189@KRBTEST.COM
I20260923 15:02:01.020586 20434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:01.011462 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.189:44251 (local address 127.0.0.1:39154)
0923 15:02:01.011537 (+ 75us) negotiation.cc:107] Waiting for socket to connect
0923 15:02:01.011549 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 15:02:01.011594 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:02:01.011851 (+ 257us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:02:01.011852 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:02:01.011990 (+ 138us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:02:01.012063 (+ 73us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:01.012065 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:01.012757 (+ 692us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:01.012759 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:01.013639 (+ 880us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:01.013647 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:01.014188 (+ 541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:01.014190 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:01.014259 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:01.015102 (+ 843us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:02:01.015107 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:02:01.016680 (+ 1573us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:02:01.018991 (+ 2311us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:02:01.018993 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:02:01.018993 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:02:01.019187 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:02:01.019535 (+ 348us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:02:01.019536 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:02:01.019536 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:02:01.019604 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:02:01.020003 (+ 399us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:02:01.020005 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:02:01.020112 (+ 107us) client_negotiation.cc:769] Sending connection context
0923 15:02:01.020352 (+ 240us) client_negotiation.cc:240] Negotiation successful
0923 15:02:01.020504 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":55}
I20260923 15:02:01.020642 20361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:01.011622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39154 (local address 127.13.75.189:44251)
0923 15:02:01.011682 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0923 15:02:01.011684 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:02:01.011693 (+ 9us) server_negotiation.cc:407] Connection header received
0923 15:02:01.011719 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:02:01.011720 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:02:01.011737 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:02:01.011799 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:02:01.012138 (+ 339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:01.012637 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:01.013819 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:01.014064 (+ 245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:01.014142 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:01.016817 (+ 2675us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:02:01.016819 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:02:01.016819 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:02:01.016828 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:02:01.018816 (+ 1988us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:02:01.019298 (+ 482us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:02:01.019299 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:02:01.019299 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:02:01.019378 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:02:01.019724 (+ 346us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:02:01.019725 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:02:01.019725 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:02:01.019862 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:02:01.019933 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0923 15:02:01.020462 (+ 529us) server_negotiation.cc:299] Negotiation successful
0923 15:02:01.020569 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260923 15:02:01.021405 20432 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.75.188" port: 33585 } has no permanent_uuid. Determining permanent_uuid...
Sep 23 15:02:01 dist-test-slave-cnb0 krb5kdc[19664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175720, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.75.188@KRBTEST.COM for oryx/127.13.75.188@KRBTEST.COM
I20260923 15:02:01.029299 20435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:01.021865 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41010 (local address 127.13.75.188:33585)
0923 15:02:01.021975 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0923 15:02:01.021977 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:02:01.021986 (+ 9us) server_negotiation.cc:407] Connection header received
0923 15:02:01.022004 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:02:01.022005 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:02:01.022015 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:02:01.022071 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:02:01.022383 (+ 312us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:01.023022 (+ 639us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:01.023769 (+ 747us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:01.023991 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:01.024160 (+ 169us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:01.026344 (+ 2184us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:02:01.026346 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:02:01.026347 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:02:01.026354 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:02:01.027923 (+ 1569us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:02:01.028327 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:02:01.028328 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:02:01.028328 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:02:01.028434 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:02:01.028715 (+ 281us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:02:01.028716 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:02:01.028716 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:02:01.028839 (+ 123us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:02:01.028902 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0923 15:02:01.029096 (+ 194us) server_negotiation.cc:299] Negotiation successful
0923 15:02:01.029213 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":48,"server-negotiator.queue_time_us":77}
I20260923 15:02:01.029300 20434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:01.021771 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.188:33585 (local address 127.0.0.1:41010)
0923 15:02:01.021858 (+ 87us) negotiation.cc:107] Waiting for socket to connect
0923 15:02:01.021872 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0923 15:02:01.021928 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:02:01.022089 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:02:01.022090 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:02:01.022234 (+ 144us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:02:01.022277 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:01.022279 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:01.023113 (+ 834us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:01.023114 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:01.023679 (+ 565us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:01.023681 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:01.024195 (+ 514us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:01.024196 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:01.024242 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:01.024866 (+ 624us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:02:01.024872 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:02:01.026249 (+ 1377us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:02:01.028028 (+ 1779us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:02:01.028030 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:02:01.028030 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:02:01.028236 (+ 206us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:02:01.028554 (+ 318us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:02:01.028555 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:02:01.028555 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:02:01.028612 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:02:01.028937 (+ 325us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:02:01.028938 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:02:01.028990 (+ 52us) client_negotiation.cc:769] Sending connection context
0923 15:02:01.029066 (+ 76us) client_negotiation.cc:240] Negotiation successful
0923 15:02:01.029165 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":59}
I20260923 15:02:01.031493 20432 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 28aa43d448d94ccdb6d8776e93de4fc9: Bootstrap starting.
I20260923 15:02:01.032367 20432 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 28aa43d448d94ccdb6d8776e93de4fc9: Neither blocks nor log segments found. Creating new log.
I20260923 15:02:01.032657 20432 log.cc:824] T 00000000000000000000000000000000 P 28aa43d448d94ccdb6d8776e93de4fc9: Log is configured to *not* fsync() on all Append() calls
I20260923 15:02:01.033495 20432 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 28aa43d448d94ccdb6d8776e93de4fc9: No bootstrap required, opened a new log
I20260923 15:02:01.035108 20432 raft_consensus.cc:348] T 00000000000000000000000000000000 P 28aa43d448d94ccdb6d8776e93de4fc9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "176378cdc0584ce69eca703063507558" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 35255 } } peers { permanent_uuid: "b8c37de502e542aab23126e30c0e3176" member_type: VOTER last_known_addr { host: "127.13.75.189" port: 44251 } } peers { permanent_uuid: "28aa43d448d94ccdb6d8776e93de4fc9" member_type: VOTER last_known_addr { host: "127.13.75.188" port: 33585 } }
I20260923 15:02:01.035251 20432 raft_consensus.cc:374] T 00000000000000000000000000000000 P 28aa43d448d94ccdb6d8776e93de4fc9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:02:01.035274 20432 raft_consensus.cc:729] T 00000000000000000000000000000000 P 28aa43d448d94ccdb6d8776e93de4fc9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 28aa43d448d94ccdb6d8776e93de4fc9, State: Initialized, Role: FOLLOWER
I20260923 15:02:01.035416 20432 consensus_queue.cc:260] T 00000000000000000000000000000000 P 28aa43d448d94ccdb6d8776e93de4fc9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "176378cdc0584ce69eca703063507558" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 35255 } } peers { permanent_uuid: "b8c37de502e542aab23126e30c0e3176" member_type: VOTER last_known_addr { host: "127.13.75.189" port: 44251 } } peers { permanent_uuid: "28aa43d448d94ccdb6d8776e93de4fc9" member_type: VOTER last_known_addr { host: "127.13.75.188" port: 33585 } }
I20260923 15:02:01.035729 20432 sys_catalog.cc:564] T 00000000000000000000000000000000 P 28aa43d448d94ccdb6d8776e93de4fc9 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 15:02:01.035732 20437 sys_catalog.cc:455] T 00000000000000000000000000000000 P 28aa43d448d94ccdb6d8776e93de4fc9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "176378cdc0584ce69eca703063507558" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 35255 } } peers { permanent_uuid: "b8c37de502e542aab23126e30c0e3176" member_type: VOTER last_known_addr { host: "127.13.75.189" port: 44251 } } peers { permanent_uuid: "28aa43d448d94ccdb6d8776e93de4fc9" member_type: VOTER last_known_addr { host: "127.13.75.188" port: 33585 } } }
I20260923 15:02:01.035902 20437 sys_catalog.cc:458] T 00000000000000000000000000000000 P 28aa43d448d94ccdb6d8776e93de4fc9 [sys.catalog]: This master's current role is: FOLLOWER
W20260923 15:02:01.037829 20448 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 28aa43d448d94ccdb6d8776e93de4fc9: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260923 15:02:01.037900 20448 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260923 15:02:01.041980 20364 master_runner.cc:428] Detected that this master 28aa43d448d94ccdb6d8776e93de4fc9 is joining an existing cluster
I20260923 15:02:01.042042 20364 master_runner.cc:432] Initiating AddMaster RPC to add 127.13.75.188:33585
I20260923 15:02:01.049981 20245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:01.043470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50794 (local address 127.13.75.190:35255)
0923 15:02:01.043538 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0923 15:02:01.043541 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:02:01.043601 (+ 60us) server_negotiation.cc:407] Connection header received
0923 15:02:01.043682 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:02:01.043683 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:02:01.043694 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:02:01.043768 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:02:01.044160 (+ 392us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:01.044665 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:01.045413 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:01.045616 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:01.045696 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:01.046969 (+ 1273us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:02:01.046971 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:02:01.046971 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:02:01.046978 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:02:01.048656 (+ 1678us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:02:01.049076 (+ 420us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:02:01.049078 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:02:01.049078 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:02:01.049146 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:02:01.049369 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:02:01.049370 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:02:01.049370 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:02:01.049507 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:02:01.049579 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0923 15:02:01.049786 (+ 207us) server_negotiation.cc:299] Negotiation successful
0923 15:02:01.049870 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260923 15:02:01.050525 20361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:01.043182 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39160 (local address 127.13.75.189:44251)
0923 15:02:01.043230 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0923 15:02:01.043232 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:02:01.043668 (+ 436us) server_negotiation.cc:407] Connection header received
0923 15:02:01.043770 (+ 102us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:02:01.043771 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:02:01.043783 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:02:01.043814 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:02:01.044358 (+ 544us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:01.045142 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:01.046220 (+ 1078us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:01.046526 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:01.046569 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:01.047672 (+ 1103us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:02:01.047674 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:02:01.047674 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:02:01.047682 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:02:01.049325 (+ 1643us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:02:01.049709 (+ 384us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:02:01.049710 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:02:01.049711 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:02:01.049769 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:02:01.049960 (+ 191us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:02:01.049961 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:02:01.049961 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:02:01.050096 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:02:01.050156 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0923 15:02:01.050337 (+ 181us) server_negotiation.cc:299] Negotiation successful
0923 15:02:01.050423 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260923 15:02:01.050659 20455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:01.043246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.189:44251 (local address 127.0.0.1:39160)
0923 15:02:01.043607 (+ 361us) negotiation.cc:107] Waiting for socket to connect
0923 15:02:01.043616 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 15:02:01.043694 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:02:01.043882 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:02:01.043941 (+ 59us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:02:01.044184 (+ 243us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:02:01.044255 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:01.044257 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:01.045242 (+ 985us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:01.045243 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:01.046116 (+ 873us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:01.046118 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:01.046648 (+ 530us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:01.046649 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:01.046711 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:01.047180 (+ 469us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:02:01.047184 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:02:01.047569 (+ 385us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:02:01.049389 (+ 1820us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:02:01.049390 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:02:01.049390 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:02:01.049615 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:02:01.049833 (+ 218us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:02:01.049834 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:02:01.049834 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:02:01.049887 (+ 53us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:02:01.050182 (+ 295us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:02:01.050183 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:02:01.050223 (+ 40us) client_negotiation.cc:769] Sending connection context
0923 15:02:01.050306 (+ 83us) client_negotiation.cc:240] Negotiation successful
0923 15:02:01.050429 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":349,"thread_start_us":93,"threads_started":1}
I20260923 15:02:01.050860 20456 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:01.043125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:35255 (local address 127.0.0.1:50794)
0923 15:02:01.043544 (+ 419us) negotiation.cc:107] Waiting for socket to connect
0923 15:02:01.043556 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 15:02:01.043618 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:02:01.043771 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:02:01.043772 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:02:01.044017 (+ 245us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:02:01.044079 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:01.044081 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:01.044753 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:01.044754 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:01.045317 (+ 563us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:01.045319 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:01.045941 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:01.045942 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:01.045998 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:01.046491 (+ 493us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:02:01.046503 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:02:01.046822 (+ 319us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:02:01.048758 (+ 1936us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:02:01.048759 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:02:01.048760 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:02:01.048993 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:02:01.049222 (+ 229us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:02:01.049223 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:02:01.049224 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:02:01.049313 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:02:01.049609 (+ 296us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:02:01.049610 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:02:01.049669 (+ 59us) client_negotiation.cc:769] Sending connection context
0923 15:02:01.050292 (+ 623us) client_negotiation.cc:240] Negotiation successful
0923 15:02:01.050502 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":387,"mutex_wait_us":28,"thread_start_us":81,"threads_started":1}
Sep 23 15:02:01 dist-test-slave-cnb0 krb5kdc[19664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175718, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.75.190@KRBTEST.COM for oryx/127.13.75.188@KRBTEST.COM
I20260923 15:02:01.063105 13614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKC4soy/build/release/bin/kudu with pid 19771
I20260923 15:02:01.066963 20437 raft_consensus.cc:482] T 00000000000000000000000000000000 P 28aa43d448d94ccdb6d8776e93de4fc9 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 15:02:01.067119 20437 raft_consensus.cc:504] T 00000000000000000000000000000000 P 28aa43d448d94ccdb6d8776e93de4fc9 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "176378cdc0584ce69eca703063507558" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 35255 } } peers { permanent_uuid: "b8c37de502e542aab23126e30c0e3176" member_type: VOTER last_known_addr { host: "127.13.75.189" port: 44251 } } peers { permanent_uuid: "28aa43d448d94ccdb6d8776e93de4fc9" member_type: VOTER last_known_addr { host: "127.13.75.188" port: 33585 } }
I20260923 15:02:01.067442 20437 leader_election.cc:288] T 00000000000000000000000000000000 P 28aa43d448d94ccdb6d8776e93de4fc9 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 176378cdc0584ce69eca703063507558 (127.13.75.190:35255), b8c37de502e542aab23126e30c0e3176 (127.13.75.189:44251)
I20260923 15:02:01.067874 20320 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "28aa43d448d94ccdb6d8776e93de4fc9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b8c37de502e542aab23126e30c0e3176" is_pre_election: true
I20260923 15:02:01.068012 20320 raft_consensus.cc:1799] T 00000000000000000000000000000000 P b8c37de502e542aab23126e30c0e3176 [term 1 LEARNER]: Handling vote request from an unknown peer 28aa43d448d94ccdb6d8776e93de4fc9
I20260923 15:02:01.068065 20320 raft_consensus.cc:2396] T 00000000000000000000000000000000 P b8c37de502e542aab23126e30c0e3176 [term 1 LEARNER]: Leader pre-election vote request: Denying vote to candidate 28aa43d448d94ccdb6d8776e93de4fc9 for term 1 because replica has last-logged OpId of term: 1 index: 5, which is greater than that of the candidate, which has last-logged OpId of term: 0 index: 0.
I20260923 15:02:01.067999 19719 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "28aa43d448d94ccdb6d8776e93de4fc9" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "176378cdc0584ce69eca703063507558" is_pre_election: true
I20260923 15:02:01.068773 20382 leader_election.cc:302] T 00000000000000000000000000000000 P 28aa43d448d94ccdb6d8776e93de4fc9 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28aa43d448d94ccdb6d8776e93de4fc9; no voters: 176378cdc0584ce69eca703063507558, b8c37de502e542aab23126e30c0e3176
I20260923 15:02:01.068996 20437 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 28aa43d448d94ccdb6d8776e93de4fc9 [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:02:01.069240 13614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKC4soy/build/release/bin/kudu
/tmp/dist-test-taskKC4soy/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.75.129:42411
--local_ip_for_outbound_sockets=127.13.75.129
--tserver_master_addrs=127.13.75.190:35255
--webserver_port=36225
--webserver_interface=127.13.75.129
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/krb5kdc/oryx_127.13.75.129.keytab
--principal=oryx/127.13.75.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.75.148:36053
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.13.75.190:35255,127.13.75.189:44251
--tserver_master_addrs=127.13.75.190:35255,127.13.75.189:44251,127.13.75.188:33585 with env {KRB5CCNAME=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260923 15:02:01.088425 20437 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 28aa43d448d94ccdb6d8776e93de4fc9 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
I20260923 15:02:01.092115 20435 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:01.043528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41022 (local address 127.13.75.188:33585)
0923 15:02:01.043567 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0923 15:02:01.043569 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:02:01.050778 (+ 7209us) server_negotiation.cc:407] Connection header received
0923 15:02:01.050861 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:02:01.050862 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:02:01.050872 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:02:01.050938 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:02:01.051280 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:01.051793 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:01.052685 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:01.052971 (+ 286us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:01.053051 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:01.089337 (+ 36286us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:02:01.089339 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:02:01.089340 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:02:01.089348 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:02:01.091057 (+ 1709us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:02:01.091413 (+ 356us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:02:01.091414 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:02:01.091414 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:02:01.091457 (+ 43us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:02:01.091617 (+ 160us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:02:01.091618 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:02:01.091618 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:02:01.091727 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:02:01.091763 (+ 36us) server_negotiation.cc:1035] Waiting for connection context
0923 15:02:01.091932 (+ 169us) server_negotiation.cc:299] Negotiation successful
0923 15:02:01.092021 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260923 15:02:01.092255 20269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:01.053029 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.188:33585 (local address 127.0.0.1:41032)
0923 15:02:01.053145 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0923 15:02:01.053155 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 15:02:01.053195 (+ 40us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:02:01.053561 (+ 366us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:02:01.053562 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:02:01.053724 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:02:01.053762 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:01.053764 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:01.054435 (+ 671us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:01.054437 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:01.055004 (+ 567us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:01.055005 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:01.055392 (+ 387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:01.055393 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:01.055460 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:01.055879 (+ 419us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:02:01.055885 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:02:01.057400 (+ 1515us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:02:01.089865 (+ 32465us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:02:01.089867 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:02:01.089867 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:02:01.090118 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:02:01.090326 (+ 208us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:02:01.090327 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:02:01.090328 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:02:01.090385 (+ 57us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:02:01.090672 (+ 287us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:02:01.090673 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:02:01.090723 (+ 50us) client_negotiation.cc:769] Sending connection context
0923 15:02:01.091996 (+ 1273us) client_negotiation.cc:240] Negotiation successful
0923 15:02:01.092145 (+ 149us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":101}
I20260923 15:02:01.092566 20457 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:01.053259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41032 (local address 127.13.75.188:33585)
0923 15:02:01.053422 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0923 15:02:01.053423 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 15:02:01.053430 (+ 7us) server_negotiation.cc:407] Connection header received
0923 15:02:01.053468 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:02:01.053468 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:02:01.053476 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:02:01.053545 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:02:01.053848 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:01.054339 (+ 491us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:01.055094 (+ 755us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:01.055310 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:01.055397 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:01.057592 (+ 2195us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:02:01.057593 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:02:01.057593 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:02:01.057600 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:02:01.089750 (+ 32150us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:02:01.090202 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:02:01.090203 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:02:01.090203 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:02:01.090259 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:02:01.090448 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:02:01.090448 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:02:01.090449 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:02:01.090587 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:02:01.090646 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0923 15:02:01.092376 (+ 1730us) server_negotiation.cc:299] Negotiation successful
0923 15:02:01.092464 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":145,"thread_start_us":112,"threads_started":1}
I20260923 15:02:01.092862 20455 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:01.043348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.188:33585 (local address 127.0.0.1:41022)
0923 15:02:01.050726 (+ 7378us) negotiation.cc:107] Waiting for socket to connect
0923 15:02:01.050737 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 15:02:01.050801 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:02:01.050995 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:02:01.050996 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:02:01.051149 (+ 153us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:02:01.051204 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:01.051207 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:01.051857 (+ 650us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:01.051858 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:01.052591 (+ 733us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:01.052593 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:01.053046 (+ 453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:01.053047 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:01.053118 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:01.087613 (+ 34495us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:02:01.087620 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:02:01.089174 (+ 1554us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:02:01.091161 (+ 1987us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:02:01.091161 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:02:01.091162 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:02:01.091344 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:02:01.091515 (+ 171us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:02:01.091515 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:02:01.091515 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:02:01.091562 (+ 47us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:02:01.091799 (+ 237us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:02:01.091800 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:02:01.091835 (+ 35us) client_negotiation.cc:769] Sending connection context
0923 15:02:01.092687 (+ 852us) client_negotiation.cc:240] Negotiation successful
0923 15:02:01.092777 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":7366,"thread_start_us":64,"threads_started":1}
I20260923 15:02:01.093119 19709 catalog_manager.cc:6979] Initiating ChangeConfig request to add master 127.13.75.188:33585: tablet_id: "00000000000000000000000000000000"
type: ADD_PEER
server {
permanent_uuid: "28aa43d448d94ccdb6d8776e93de4fc9"
member_type: NON_VOTER
last_known_addr {
host: "127.13.75.188"
port: 33585
}
attrs {
promote: true
}
}
dest_uuid: "176378cdc0584ce69eca703063507558"
cas_config_opid_index: 5
I20260923 15:02:01.093370 19709 consensus_queue.cc:237] T 00000000000000000000000000000000 P 176378cdc0584ce69eca703063507558 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 5, Committed index: 5, Last appended: 1.5, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "176378cdc0584ce69eca703063507558" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 35255 } } peers { permanent_uuid: "b8c37de502e542aab23126e30c0e3176" member_type: NON_VOTER last_known_addr { host: "127.13.75.189" port: 44251 } attrs { promote: true } } peers { permanent_uuid: "28aa43d448d94ccdb6d8776e93de4fc9" member_type: NON_VOTER last_known_addr { host: "127.13.75.188" port: 33585 } attrs { promote: true } }
I20260923 15:02:01.093873 20273 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 176378cdc0584ce69eca703063507558 [term 1 LEADER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 28aa43d448d94ccdb6d8776e93de4fc9 (127.13.75.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "176378cdc0584ce69eca703063507558" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 35255 } } peers { permanent_uuid: "b8c37de502e542aab23126e30c0e3176" member_type: NON_VOTER last_known_addr { host: "127.13.75.189" port: 44251 } attrs { promote: true } } peers { permanent_uuid: "28aa43d448d94ccdb6d8776e93de4fc9" member_type: NON_VOTER last_known_addr { host: "127.13.75.188" port: 33585 } attrs { promote: true } } }
I20260923 15:02:01.095144 20273 catalog_manager.cc:6983] Successfully completed master ChangeConfig request to add master 127.13.75.188:33585
I20260923 15:02:01.095239 20273 sys_catalog.cc:455] T 00000000000000000000000000000000 P 176378cdc0584ce69eca703063507558 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "176378cdc0584ce69eca703063507558" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "176378cdc0584ce69eca703063507558" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 35255 } } peers { permanent_uuid: "b8c37de502e542aab23126e30c0e3176" member_type: NON_VOTER last_known_addr { host: "127.13.75.189" port: 44251 } attrs { promote: true } } peers { permanent_uuid: "28aa43d448d94ccdb6d8776e93de4fc9" member_type: NON_VOTER last_known_addr { host: "127.13.75.188" port: 33585 } attrs { promote: true } } }
I20260923 15:02:01.095320 20273 sys_catalog.cc:458] T 00000000000000000000000000000000 P 176378cdc0584ce69eca703063507558 [sys.catalog]: This master's current role is: LEADER
I20260923 15:02:01.095407 20364 master.cc:537] Master@127.13.75.188:33585 shutting down...
W20260923 15:02:01.096000 19694 consensus_peers.cc:487] T 00000000000000000000000000000000 P 176378cdc0584ce69eca703063507558 -> Peer 28aa43d448d94ccdb6d8776e93de4fc9 (127.13.75.188:33585): Couldn't send request to peer 28aa43d448d94ccdb6d8776e93de4fc9. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260923 15:02:01.099144 20269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:01.094135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.189:44251 (local address 127.0.0.1:39170)
0923 15:02:01.094468 (+ 333us) negotiation.cc:107] Waiting for socket to connect
0923 15:02:01.094478 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 15:02:01.094669 (+ 191us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:02:01.094990 (+ 321us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:02:01.094991 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:02:01.094993 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:02:01.095052 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:01.095054 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:01.096476 (+ 1422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:01.096478 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:01.097840 (+ 1362us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:01.097845 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:01.098811 (+ 966us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:01.098813 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:01.098912 (+ 99us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:01.098945 (+ 33us) client_negotiation.cc:769] Sending connection context
0923 15:02:01.098997 (+ 52us) client_negotiation.cc:240] Negotiation successful
0923 15:02:01.099056 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":315}
I20260923 15:02:01.099567 20361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:01.094260 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39170 (local address 127.13.75.189:44251)
0923 15:02:01.094305 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0923 15:02:01.094307 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:02:01.094773 (+ 466us) server_negotiation.cc:407] Connection header received
0923 15:02:01.094806 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:02:01.094808 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:02:01.094834 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:02:01.094896 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:02:01.095139 (+ 243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:01.095996 (+ 857us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:01.097962 (+ 1966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:01.098621 (+ 659us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:01.098674 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:01.098744 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0923 15:02:01.099447 (+ 703us) server_negotiation.cc:299] Negotiation successful
0923 15:02:01.099499 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260923 15:02:01.100049 20320 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b8c37de502e542aab23126e30c0e3176 [term 1 LEARNER]: Refusing update from remote peer 176378cdc0584ce69eca703063507558: Log matching property violated. Preceding OpId in replica: term: 1 index: 5. Preceding OpId from leader: term: 1 index: 6. (index mismatch)
I20260923 15:02:01.100277 20271 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 176378cdc0584ce69eca703063507558 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8c37de502e542aab23126e30c0e3176" member_type: NON_VOTER last_known_addr { host: "127.13.75.189" port: 44251 } attrs { promote: true }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 6, Last known committed idx: 5, Time since last communication: 0.000s
I20260923 15:02:01.100946 20320 raft_consensus.cc:2932] T 00000000000000000000000000000000 P b8c37de502e542aab23126e30c0e3176 [term 1 LEARNER]: Committing config change with OpId 1.6: config changed from index 5 to 6, NON_VOTER 28aa43d448d94ccdb6d8776e93de4fc9 (127.13.75.188) added. New config: { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "176378cdc0584ce69eca703063507558" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 35255 } } peers { permanent_uuid: "b8c37de502e542aab23126e30c0e3176" member_type: NON_VOTER last_known_addr { host: "127.13.75.189" port: 44251 } attrs { promote: true } } peers { permanent_uuid: "28aa43d448d94ccdb6d8776e93de4fc9" member_type: NON_VOTER last_known_addr { host: "127.13.75.188" port: 33585 } attrs { promote: true } } }
I20260923 15:02:01.102243 20364 raft_consensus.cc:2229] T 00000000000000000000000000000000 P 28aa43d448d94ccdb6d8776e93de4fc9 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 15:02:01.102329 20364 raft_consensus.cc:2258] T 00000000000000000000000000000000 P 28aa43d448d94ccdb6d8776e93de4fc9 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 15:02:01.102367 20364 tablet_replica.cc:330] T 00000000000000000000000000000000 P 28aa43d448d94ccdb6d8776e93de4fc9: stopping tablet replica
I20260923 15:02:01.102676 20271 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 176378cdc0584ce69eca703063507558: attempting to promote NON_VOTER b8c37de502e542aab23126e30c0e3176 to VOTER
I20260923 15:02:01.102802 20349 sys_catalog.cc:455] T 00000000000000000000000000000000 P b8c37de502e542aab23126e30c0e3176 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 176378cdc0584ce69eca703063507558. Latest consensus state: current_term: 1 leader_uuid: "176378cdc0584ce69eca703063507558" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "176378cdc0584ce69eca703063507558" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 35255 } } peers { permanent_uuid: "b8c37de502e542aab23126e30c0e3176" member_type: NON_VOTER last_known_addr { host: "127.13.75.189" port: 44251 } attrs { promote: true } } peers { permanent_uuid: "28aa43d448d94ccdb6d8776e93de4fc9" member_type: NON_VOTER last_known_addr { host: "127.13.75.188" port: 33585 } attrs { promote: true } } }
I20260923 15:02:01.102941 20271 consensus_queue.cc:237] T 00000000000000000000000000000000 P 176378cdc0584ce69eca703063507558 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 6, Committed index: 6, Last appended: 1.6, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "176378cdc0584ce69eca703063507558" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 35255 } } peers { permanent_uuid: "b8c37de502e542aab23126e30c0e3176" member_type: VOTER last_known_addr { host: "127.13.75.189" port: 44251 } attrs { promote: false } } peers { permanent_uuid: "28aa43d448d94ccdb6d8776e93de4fc9" member_type: NON_VOTER last_known_addr { host: "127.13.75.188" port: 33585 } attrs { promote: true } }
I20260923 15:02:01.103011 20349 sys_catalog.cc:458] T 00000000000000000000000000000000 P b8c37de502e542aab23126e30c0e3176 [sys.catalog]: This master's current role is: LEARNER
I20260923 15:02:01.103061 20349 sys_catalog.cc:455] T 00000000000000000000000000000000 P b8c37de502e542aab23126e30c0e3176 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "176378cdc0584ce69eca703063507558" committed_config { opid_index: 6 OBSOLETE_local: true peers { permanent_uuid: "176378cdc0584ce69eca703063507558" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 35255 } } peers { permanent_uuid: "b8c37de502e542aab23126e30c0e3176" member_type: NON_VOTER last_known_addr { host: "127.13.75.189" port: 44251 } attrs { promote: true } } peers { permanent_uuid: "28aa43d448d94ccdb6d8776e93de4fc9" member_type: NON_VOTER last_known_addr { host: "127.13.75.188" port: 33585 } attrs { promote: true } } }
I20260923 15:02:01.103116 20349 sys_catalog.cc:458] T 00000000000000000000000000000000 P b8c37de502e542aab23126e30c0e3176 [sys.catalog]: This master's current role is: LEARNER
I20260923 15:02:01.103398 20320 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b8c37de502e542aab23126e30c0e3176 [term 1 LEARNER]: Refusing update from remote peer 176378cdc0584ce69eca703063507558: Log matching property violated. Preceding OpId in replica: term: 1 index: 6. Preceding OpId from leader: term: 1 index: 7. (index mismatch)
I20260923 15:02:01.103641 20273 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 176378cdc0584ce69eca703063507558 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8c37de502e542aab23126e30c0e3176" member_type: VOTER last_known_addr { host: "127.13.75.189" port: 44251 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 7, Last known committed idx: 6, Time since last communication: 0.000s
W20260923 15:02:01.103722 19694 consensus_peers.cc:487] T 00000000000000000000000000000000 P 176378cdc0584ce69eca703063507558 -> Peer 28aa43d448d94ccdb6d8776e93de4fc9 (127.13.75.188:33585): Couldn't send request to peer 28aa43d448d94ccdb6d8776e93de4fc9. Status: Remote error: Service unavailable: service kudu.consensus.ConsensusService not registered on Master. This is attempt 1: this message will repeat every 5th retry.
I20260923 15:02:01.104770 20273 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 176378cdc0584ce69eca703063507558 [term 1 LEADER]: Committing config change with OpId 1.7: config changed from index 6 to 7, b8c37de502e542aab23126e30c0e3176 (127.13.75.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "176378cdc0584ce69eca703063507558" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 35255 } } peers { permanent_uuid: "b8c37de502e542aab23126e30c0e3176" member_type: VOTER last_known_addr { host: "127.13.75.189" port: 44251 } attrs { promote: false } } peers { permanent_uuid: "28aa43d448d94ccdb6d8776e93de4fc9" member_type: NON_VOTER last_known_addr { host: "127.13.75.188" port: 33585 } attrs { promote: true } } }
I20260923 15:02:01.105966 20462 sys_catalog.cc:455] T 00000000000000000000000000000000 P 176378cdc0584ce69eca703063507558 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "176378cdc0584ce69eca703063507558" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "176378cdc0584ce69eca703063507558" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 35255 } } peers { permanent_uuid: "b8c37de502e542aab23126e30c0e3176" member_type: VOTER last_known_addr { host: "127.13.75.189" port: 44251 } attrs { promote: false } } peers { permanent_uuid: "28aa43d448d94ccdb6d8776e93de4fc9" member_type: NON_VOTER last_known_addr { host: "127.13.75.188" port: 33585 } attrs { promote: true } } }
I20260923 15:02:01.106053 20462 sys_catalog.cc:458] T 00000000000000000000000000000000 P 176378cdc0584ce69eca703063507558 [sys.catalog]: This master's current role is: LEADER
I20260923 15:02:01.106318 20271 sys_catalog.cc:455] T 00000000000000000000000000000000 P 176378cdc0584ce69eca703063507558 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "176378cdc0584ce69eca703063507558" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "176378cdc0584ce69eca703063507558" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 35255 } } peers { permanent_uuid: "b8c37de502e542aab23126e30c0e3176" member_type: VOTER last_known_addr { host: "127.13.75.189" port: 44251 } attrs { promote: false } } peers { permanent_uuid: "28aa43d448d94ccdb6d8776e93de4fc9" member_type: NON_VOTER last_known_addr { host: "127.13.75.188" port: 33585 } attrs { promote: true } } }
I20260923 15:02:01.106381 20271 sys_catalog.cc:458] T 00000000000000000000000000000000 P 176378cdc0584ce69eca703063507558 [sys.catalog]: This master's current role is: LEADER
I20260923 15:02:01.106681 20320 raft_consensus.cc:2932] T 00000000000000000000000000000000 P b8c37de502e542aab23126e30c0e3176 [term 1 FOLLOWER]: Committing config change with OpId 1.7: config changed from index 6 to 7, b8c37de502e542aab23126e30c0e3176 (127.13.75.189) changed from NON_VOTER to VOTER. New config: { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "176378cdc0584ce69eca703063507558" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 35255 } } peers { permanent_uuid: "b8c37de502e542aab23126e30c0e3176" member_type: VOTER last_known_addr { host: "127.13.75.189" port: 44251 } attrs { promote: false } } peers { permanent_uuid: "28aa43d448d94ccdb6d8776e93de4fc9" member_type: NON_VOTER last_known_addr { host: "127.13.75.188" port: 33585 } attrs { promote: true } } }
I20260923 15:02:01.107977 20461 sys_catalog.cc:455] T 00000000000000000000000000000000 P b8c37de502e542aab23126e30c0e3176 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "176378cdc0584ce69eca703063507558" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "176378cdc0584ce69eca703063507558" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 35255 } } peers { permanent_uuid: "b8c37de502e542aab23126e30c0e3176" member_type: VOTER last_known_addr { host: "127.13.75.189" port: 44251 } attrs { promote: false } } peers { permanent_uuid: "28aa43d448d94ccdb6d8776e93de4fc9" member_type: NON_VOTER last_known_addr { host: "127.13.75.188" port: 33585 } attrs { promote: true } } }
I20260923 15:02:01.108079 20461 sys_catalog.cc:458] T 00000000000000000000000000000000 P b8c37de502e542aab23126e30c0e3176 [sys.catalog]: This master's current role is: FOLLOWER
I20260923 15:02:01.116763 20364 master.cc:552] Master@127.13.75.188:33585 shutdown complete.
I20260923 15:02:01.116828 20364 master_runner.cc:305] Clearing existing system tablet
I20260923 15:02:01.116961 20364 fs_manager.cc:359] Metadata directory not provided
I20260923 15:02:01.117012 20364 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:02:01.117805 20364 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:02:01.118403 20465 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:02:01.118561 20364 fs_report.cc:383] FS layout report
--------------------
wal directory:
metadata directory:
1 data directories: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:02:01.118634 20364 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260923 15:02:01.118659 20364 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/minicluster-data/master-2/data,/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/minicluster-data/master-2/wal
uuid: "28aa43d448d94ccdb6d8776e93de4fc9"
format_stamp: "Formatted at 2026-09-23 15:02:00 on dist-test-slave-cnb0"
server_key: "de9a8214d89a2004e0f482bbc54b2df1"
server_key_iv: "f38f585feaa2efbcc266e0e2a47ff902"
server_key_version: "encryptionkey@0"
I20260923 15:02:01.119442 20364 ts_tablet_manager.cc:1905] T 00000000000000000000000000000000 P 28aa43d448d94ccdb6d8776e93de4fc9: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 15:02:01.122089 20364 ts_tablet_manager.cc:1918] T 00000000000000000000000000000000 P 28aa43d448d94ccdb6d8776e93de4fc9: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId unknown
I20260923 15:02:01.122190 20364 log.cc:1196] T 00000000000000000000000000000000 P 28aa43d448d94ccdb6d8776e93de4fc9: Deleting WAL directory at /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/minicluster-data/master-2/wal/wals/00000000000000000000000000000000
I20260923 15:02:01.122393 20364 ts_tablet_manager.cc:1939] T 00000000000000000000000000000000 P 28aa43d448d94ccdb6d8776e93de4fc9: Deleting consensus metadata
I20260923 15:02:01.122516 20364 master_runner.cc:315] Copying system tablet from 127.13.75.190:35255
I20260923 15:02:01.123224 20364 tablet_copy_client.cc:316] T 00000000000000000000000000000000 P 28aa43d448d94ccdb6d8776e93de4fc9: tablet copy: Beginning tablet copy session from remote peer at address 127.13.75.190:35255
I20260923 15:02:01.131158 20245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:01.123677 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50796 (local address 127.13.75.190:35255)
0923 15:02:01.123730 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0923 15:02:01.123732 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:02:01.123795 (+ 63us) server_negotiation.cc:407] Connection header received
0923 15:02:01.123956 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:02:01.123958 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:02:01.123978 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:02:01.124065 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:02:01.124462 (+ 397us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:01.125211 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:01.126266 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:01.126560 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:01.126645 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:01.128008 (+ 1363us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:02:01.128010 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:02:01.128010 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:02:01.128019 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:02:01.129760 (+ 1741us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:02:01.130203 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:02:01.130204 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:02:01.130205 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:02:01.130274 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:02:01.130502 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:02:01.130503 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:02:01.130503 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:02:01.130666 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:02:01.130744 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0923 15:02:01.130952 (+ 208us) server_negotiation.cc:299] Negotiation successful
0923 15:02:01.131043 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260923 15:02:01.131248 20471 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:01.123592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:35255 (local address 127.0.0.1:50796)
0923 15:02:01.123722 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0923 15:02:01.123733 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 15:02:01.123873 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:02:01.124085 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:02:01.124086 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:02:01.124289 (+ 203us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:02:01.124359 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:01.124361 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:01.125311 (+ 950us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:01.125312 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:01.126157 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:01.126160 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:01.126675 (+ 515us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:01.126676 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:01.126743 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:01.127460 (+ 717us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:02:01.127466 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:02:01.127886 (+ 420us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:02:01.129864 (+ 1978us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:02:01.129866 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:02:01.129867 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:02:01.130110 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:02:01.130350 (+ 240us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:02:01.130351 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:02:01.130352 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:02:01.130427 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:02:01.130743 (+ 316us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:02:01.130744 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:02:01.130800 (+ 56us) client_negotiation.cc:769] Sending connection context
0923 15:02:01.130875 (+ 75us) client_negotiation.cc:240] Negotiation successful
0923 15:02:01.131134 (+ 259us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":72,"threads_started":1}
I20260923 15:02:01.131958 19729 tablet_copy_service.cc:140] P 176378cdc0584ce69eca703063507558: Received BeginTabletCopySession request for tablet 00000000000000000000000000000000 from peer 28aa43d448d94ccdb6d8776e93de4fc9 ({username='oryx', principal='oryx/127.13.75.188@KRBTEST.COM'} at 127.0.0.1:50796)
I20260923 15:02:01.132035 19729 tablet_copy_service.cc:161] P 176378cdc0584ce69eca703063507558: Beginning new tablet copy session on tablet 00000000000000000000000000000000 from peer 28aa43d448d94ccdb6d8776e93de4fc9 at {username='oryx', principal='oryx/127.13.75.188@KRBTEST.COM'} at 127.0.0.1:50796: session id = 28aa43d448d94ccdb6d8776e93de4fc9-00000000000000000000000000000000
I20260923 15:02:01.132673 19729 tablet_copy_source_session.cc:205] T 00000000000000000000000000000000 P 176378cdc0584ce69eca703063507558: Tablet Copy: opened 0 blocks and 1 log segments
I20260923 15:02:01.133242 20364 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total
I20260923 15:02:01.134778 20364 tablet_copy_client.cc:791] T 00000000000000000000000000000000 P 28aa43d448d94ccdb6d8776e93de4fc9: tablet copy: Starting download of 0 data blocks...
I20260923 15:02:01.134995 20364 tablet_copy_client.cc:655] T 00000000000000000000000000000000 P 28aa43d448d94ccdb6d8776e93de4fc9: tablet copy: Starting download of 1 WAL segments...
I20260923 15:02:01.135859 20364 tablet_copy_client.cc:531] T 00000000000000000000000000000000 P 28aa43d448d94ccdb6d8776e93de4fc9: tablet copy: Tablet Copy complete. Replacing tablet superblock.
I20260923 15:02:01.137043 19729 tablet_copy_service.cc:342] P 176378cdc0584ce69eca703063507558: Request end of tablet copy session 28aa43d448d94ccdb6d8776e93de4fc9-00000000000000000000000000000000 received from {username='oryx', principal='oryx/127.13.75.188@KRBTEST.COM'} at 127.0.0.1:50796
I20260923 15:02:01.137108 19729 tablet_copy_service.cc:433] P 176378cdc0584ce69eca703063507558: ending tablet copy session 28aa43d448d94ccdb6d8776e93de4fc9-00000000000000000000000000000000 on tablet 00000000000000000000000000000000 with peer 28aa43d448d94ccdb6d8776e93de4fc9
I20260923 15:02:01.140851 20364 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:02:01.142139 20472 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:02:01.144627 20473 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:02:01.144773 20475 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:02:01.148460 20364 server_base.cc:1034] running on GCE node
I20260923 15:02:01.148617 20364 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:02:01.148799 20364 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:02:01.149948 20364 hybrid_clock.cc:648] HybridClock initialized: now 1790175721149925 us; error 38 us; skew 500 ppm
Sep 23 15:02:01 dist-test-slave-cnb0 krb5kdc[19664](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175721, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.75.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:02:01.151582 20364 init.cc:377] Logged in from keytab as oryx/127.13.75.188@KRBTEST.COM (short username oryx)
I20260923 15:02:01.151866 20364 webserver.cc:460] Webserver started at http://127.13.75.188:44179/ using document root <none> and password file <none>
I20260923 15:02:01.151995 20364 fs_manager.cc:359] Metadata directory not provided
I20260923 15:02:01.152031 20364 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:02:01.152840 20364 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:02:01.153455 20482 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:02:01.153578 20364 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.000s
I20260923 15:02:01.153620 20364 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/minicluster-data/master-2/data,/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/minicluster-data/master-2/wal
uuid: "28aa43d448d94ccdb6d8776e93de4fc9"
format_stamp: "Formatted at 2026-09-23 15:02:00 on dist-test-slave-cnb0"
server_key: "de9a8214d89a2004e0f482bbc54b2df1"
server_key_iv: "f38f585feaa2efbcc266e0e2a47ff902"
server_key_version: "encryptionkey@0"
I20260923 15:02:01.153684 20364 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:02:01.166095 20364 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:02:01.168449 20364 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:02:01.171886 20364 rpc_server.cc:307] RPC server started. Bound to: 127.13.75.188:33585
I20260923 15:02:01.171922 20534 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.75.188:33585 every 8 connection(s)
I20260923 15:02:01.172345 20364 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/minicluster-data/master-2/data/info.pb
I20260923 15:02:01.175782 20535 sys_catalog.cc:263] Verifying existing consensus state
I20260923 15:02:01.176381 20535 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 28aa43d448d94ccdb6d8776e93de4fc9: Bootstrap starting.
I20260923 15:02:01.179658 20535 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 28aa43d448d94ccdb6d8776e93de4fc9: Bootstrap replayed 1/1 log segments. Stats: ops{read=7 overwritten=0 applied=7 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260923 15:02:01.179904 20535 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 28aa43d448d94ccdb6d8776e93de4fc9: Bootstrap complete.
I20260923 15:02:01.180131 20535 raft_consensus.cc:348] T 00000000000000000000000000000000 P 28aa43d448d94ccdb6d8776e93de4fc9 [term 1 LEARNER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "176378cdc0584ce69eca703063507558" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 35255 } } peers { permanent_uuid: "b8c37de502e542aab23126e30c0e3176" member_type: VOTER last_known_addr { host: "127.13.75.189" port: 44251 } attrs { promote: false } } peers { permanent_uuid: "28aa43d448d94ccdb6d8776e93de4fc9" member_type: NON_VOTER last_known_addr { host: "127.13.75.188" port: 33585 } attrs { promote: true } }
I20260923 15:02:01.180226 20535 raft_consensus.cc:729] T 00000000000000000000000000000000 P 28aa43d448d94ccdb6d8776e93de4fc9 [term 1 LEARNER]: Becoming Follower/Learner. State: Replica: 28aa43d448d94ccdb6d8776e93de4fc9, State: Initialized, Role: LEARNER
I20260923 15:02:01.180260 20535 consensus_queue.cc:260] T 00000000000000000000000000000000 P 28aa43d448d94ccdb6d8776e93de4fc9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 7, Last appended: 1.7, Last appended by leader: 7, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "176378cdc0584ce69eca703063507558" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 35255 } } peers { permanent_uuid: "b8c37de502e542aab23126e30c0e3176" member_type: VOTER last_known_addr { host: "127.13.75.189" port: 44251 } attrs { promote: false } } peers { permanent_uuid: "28aa43d448d94ccdb6d8776e93de4fc9" member_type: NON_VOTER last_known_addr { host: "127.13.75.188" port: 33585 } attrs { promote: true } }
I20260923 15:02:01.180436 20535 sys_catalog.cc:564] T 00000000000000000000000000000000 P 28aa43d448d94ccdb6d8776e93de4fc9 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 15:02:01.180449 20541 sys_catalog.cc:455] T 00000000000000000000000000000000 P 28aa43d448d94ccdb6d8776e93de4fc9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "176378cdc0584ce69eca703063507558" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 35255 } } peers { permanent_uuid: "b8c37de502e542aab23126e30c0e3176" member_type: VOTER last_known_addr { host: "127.13.75.189" port: 44251 } attrs { promote: false } } peers { permanent_uuid: "28aa43d448d94ccdb6d8776e93de4fc9" member_type: NON_VOTER last_known_addr { host: "127.13.75.188" port: 33585 } attrs { promote: true } } }
I20260923 15:02:01.180650 20541 sys_catalog.cc:458] T 00000000000000000000000000000000 P 28aa43d448d94ccdb6d8776e93de4fc9 [sys.catalog]: This master's current role is: LEARNER
W20260923 15:02:01.181211 20458 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:02:01.181433 20458 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:02:01.181478 20458 flags.cc:425] Enabled unsafe flag: --never_fsync=true
I20260923 15:02:01.181951 20552 catalog_manager.cc:1259] Loaded cluster ID: 4b5be46274e44cd197c2c043e352da13
I20260923 15:02:01.182091 20552 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 28aa43d448d94ccdb6d8776e93de4fc9: loading cluster ID for follower catalog manager: success
W20260923 15:02:01.183446 20458 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
I20260923 15:02:01.183462 20552 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 28aa43d448d94ccdb6d8776e93de4fc9: acquiring CA information for follower catalog manager: success
W20260923 15:02:01.183511 20458 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:02:01.183557 20458 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.75.129
I20260923 15:02:01.183773 20552 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 28aa43d448d94ccdb6d8776e93de4fc9: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260923 15:02:01.185459 20458 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.75.148:36053
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/krb5kdc/oryx_127.13.75.129.keytab
--principal=oryx/127.13.75.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.75.129:42411
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.75.129
--webserver_port=36225
--tserver_master_addrs=127.13.75.190:35255,127.13.75.189:44251,127.13.75.188:33585
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20458
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.75.129
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:02:01.185721 20458 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:02:01.185976 20458 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:02:01.191623 20554 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:02:01.191859 20458 server_base.cc:1034] running on GCE node
W20260923 15:02:01.191673 20555 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:02:01.191641 20557 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:02:01.192314 20458 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:02:01.192636 20458 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:02:01.193806 20458 hybrid_clock.cc:648] HybridClock initialized: now 1790175721193777 us; error 56 us; skew 500 ppm
Sep 23 15:02:01 dist-test-slave-cnb0 krb5kdc[19664](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175721, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.75.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:02:01.197121 20458 init.cc:377] Logged in from keytab as oryx/127.13.75.129@KRBTEST.COM (short username oryx)
I20260923 15:02:01.197651 20458 webserver.cc:460] Webserver started at http://127.13.75.129:36225/ using document root <none> and password file <none>
I20260923 15:02:01.197896 20458 fs_manager.cc:359] Metadata directory not provided
I20260923 15:02:01.197985 20458 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:02:01.199566 20458 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 15:02:01.200389 20564 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:02:01.200613 20458 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:02:01.200726 20458 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/minicluster-data/ts-0/wal
uuid: "e9ef8b623efa4412ae2bfff361a0a8a8"
format_stamp: "Formatted at 2026-09-23 15:01:58 on dist-test-slave-cnb0"
server_key: "69cfdf9c2f665a3d6ab4197dc11358d8"
server_key_iv: "0da10c81dae47b6d40a3ea133babe112"
server_key_version: "encryptionkey@0"
I20260923 15:02:01.201125 20458 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:02:01.215034 20458 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:02:01.217789 20458 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:02:01.217993 20458 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:02:01.218264 20458 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 15:02:01.218617 20458 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 15:02:01.218675 20458 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 15:02:01.218719 20458 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 15:02:01.218747 20458 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 15:02:01.224830 20458 rpc_server.cc:307] RPC server started. Bound to: 127.13.75.129:42411
I20260923 15:02:01.224910 20677 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.75.129:42411 every 8 connection(s)
I20260923 15:02:01.225274 20458 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/minicluster-data/ts-0/data/info.pb
I20260923 15:02:01.226128 13614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKC4soy/build/release/bin/kudu as pid 20458
I20260923 15:02:01.226284 13614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKC4soy/build/release/bin/kudu with pid 19905
I20260923 15:02:01.233496 13614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKC4soy/build/release/bin/kudu
/tmp/dist-test-taskKC4soy/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.75.130:42895
--local_ip_for_outbound_sockets=127.13.75.130
--tserver_master_addrs=127.13.75.190:35255
--webserver_port=44197
--webserver_interface=127.13.75.130
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/krb5kdc/oryx_127.13.75.130.keytab
--principal=oryx/127.13.75.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.75.148:36053
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.13.75.190:35255,127.13.75.189:44251
--tserver_master_addrs=127.13.75.190:35255,127.13.75.189:44251,127.13.75.188:33585 with env {KRB5CCNAME=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 23 15:02:01 dist-test-slave-cnb0 krb5kdc[19664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175721, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.75.129@KRBTEST.COM for oryx/127.13.75.189@KRBTEST.COM
Sep 23 15:02:01 dist-test-slave-cnb0 krb5kdc[19664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175721, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.75.129@KRBTEST.COM for oryx/127.13.75.188@KRBTEST.COM
I20260923 15:02:01.239943 20361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:01.226871 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.129:45737 (local address 127.13.75.189:44251)
0923 15:02:01.227061 (+ 190us) server_negotiation.cc:206] Beginning negotiation
0923 15:02:01.227063 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:02:01.227289 (+ 226us) server_negotiation.cc:407] Connection header received
0923 15:02:01.228888 (+ 1599us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:02:01.228890 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:02:01.228903 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:02:01.228935 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:02:01.229953 (+ 1018us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:01.230571 (+ 618us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:01.231719 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:01.231940 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:01.232033 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:01.236336 (+ 4303us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:02:01.236338 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:02:01.236339 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:02:01.236348 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:02:01.238108 (+ 1760us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:02:01.238690 (+ 582us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:02:01.238691 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:02:01.238692 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:02:01.238755 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:02:01.239104 (+ 349us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:02:01.239105 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:02:01.239105 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:02:01.239256 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:02:01.239315 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0923 15:02:01.239756 (+ 441us) server_negotiation.cc:299] Negotiation successful
0923 15:02:01.239848 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":167}
Sep 23 15:02:01 dist-test-slave-cnb0 krb5kdc[19664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175721, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.75.129@KRBTEST.COM for oryx/127.13.75.190@KRBTEST.COM
I20260923 15:02:01.240460 20683 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:01.226156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.189:44251 (local address 127.13.75.129:45737)
0923 15:02:01.227142 (+ 986us) negotiation.cc:107] Waiting for socket to connect
0923 15:02:01.227201 (+ 59us) client_negotiation.cc:174] Beginning negotiation
0923 15:02:01.228748 (+ 1547us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:02:01.229135 (+ 387us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:02:01.229140 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:02:01.229567 (+ 427us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:02:01.229867 (+ 300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:01.229873 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:01.230677 (+ 804us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:01.230678 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:01.231621 (+ 943us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:01.231624 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:01.232047 (+ 423us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:01.232048 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:01.232217 (+ 169us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:01.233951 (+ 1734us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:02:01.233961 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:02:01.236226 (+ 2265us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:02:01.238209 (+ 1983us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:02:01.238219 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:02:01.238221 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:02:01.238599 (+ 378us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:02:01.238829 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:02:01.238829 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:02:01.238830 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:02:01.239008 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:02:01.239372 (+ 364us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:02:01.239381 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:02:01.239578 (+ 197us) client_negotiation.cc:769] Sending connection context
0923 15:02:01.240085 (+ 507us) client_negotiation.cc:240] Negotiation successful
0923 15:02:01.240205 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":941,"mutex_wait_us":2,"thread_start_us":103,"threads_started":1}
I20260923 15:02:01.240828 20682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:01.226154 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.129:40673 (local address 127.13.75.188:33585)
0923 15:02:01.226302 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0923 15:02:01.226304 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:02:01.230336 (+ 4032us) server_negotiation.cc:407] Connection header received
0923 15:02:01.230377 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:02:01.230378 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:02:01.230394 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:02:01.230468 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:02:01.230995 (+ 527us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:01.232102 (+ 1107us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:01.233288 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:01.233565 (+ 277us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:01.233630 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:01.236679 (+ 3049us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:02:01.236681 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:02:01.236681 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:02:01.236689 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:02:01.238364 (+ 1675us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:02:01.238877 (+ 513us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:02:01.238878 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:02:01.238879 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:02:01.238966 (+ 87us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:02:01.239162 (+ 196us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:02:01.239163 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:02:01.239163 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:02:01.239289 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:02:01.239345 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0923 15:02:01.240350 (+ 1005us) server_negotiation.cc:299] Negotiation successful
0923 15:02:01.240442 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":115,"thread_start_us":61,"threads_started":1}
I20260923 15:02:01.240918 20680 heartbeater.cc:347] Connected to a master server at 127.13.75.189:44251
I20260923 15:02:01.241022 20680 heartbeater.cc:464] Registering TS with master...
I20260923 15:02:01.241209 20680 heartbeater.cc:511] Master 127.13.75.189:44251 requested a full tablet report, sending...
I20260923 15:02:01.241680 20310 ts_manager.cc:194] Registered new tserver with Master: e9ef8b623efa4412ae2bfff361a0a8a8 (127.13.75.129:42411)
I20260923 15:02:01.242514 20685 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:01.226156 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.188:33585 (local address 127.13.75.129:40673)
0923 15:02:01.230229 (+ 4073us) negotiation.cc:107] Waiting for socket to connect
0923 15:02:01.230245 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0923 15:02:01.230322 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:02:01.230503 (+ 181us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:02:01.230504 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:02:01.230749 (+ 245us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:02:01.230858 (+ 109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:01.230860 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:01.232191 (+ 1331us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:01.232192 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:01.233185 (+ 993us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:01.233189 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:01.233696 (+ 507us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:01.233697 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:01.233762 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:01.234164 (+ 402us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:02:01.234169 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:02:01.236586 (+ 2417us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:02:01.238449 (+ 1863us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:02:01.238450 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:02:01.238450 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:02:01.238801 (+ 351us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:02:01.239035 (+ 234us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:02:01.239036 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:02:01.239036 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:02:01.239094 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:02:01.239372 (+ 278us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:02:01.239381 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:02:01.239578 (+ 197us) client_negotiation.cc:769] Sending connection context
0923 15:02:01.239701 (+ 123us) client_negotiation.cc:240] Negotiation successful
0923 15:02:01.239873 (+ 172us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":4035,"mutex_wait_us":30,"thread_start_us":109,"threads_started":1}
I20260923 15:02:01.243121 20679 heartbeater.cc:347] Connected to a master server at 127.13.75.188:33585
I20260923 15:02:01.243188 20679 heartbeater.cc:464] Registering TS with master...
I20260923 15:02:01.243264 20679 heartbeater.cc:511] Master 127.13.75.188:33585 requested a full tablet report, sending...
I20260923 15:02:01.243654 20499 ts_manager.cc:194] Registered new tserver with Master: e9ef8b623efa4412ae2bfff361a0a8a8 (127.13.75.129:42411)
I20260923 15:02:01.244191 20245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:01.227442 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.129:56635 (local address 127.13.75.190:35255)
0923 15:02:01.227493 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0923 15:02:01.227495 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:02:01.233308 (+ 5813us) server_negotiation.cc:407] Connection header received
0923 15:02:01.233789 (+ 481us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:02:01.233790 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:02:01.233805 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:02:01.233839 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:02:01.234624 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:01.235420 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:01.236521 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:01.237868 (+ 1347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:01.237925 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:01.240632 (+ 2707us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:02:01.240634 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:02:01.240634 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:02:01.240644 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:02:01.242683 (+ 2039us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:02:01.243193 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:02:01.243195 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:02:01.243195 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:02:01.243266 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:02:01.243642 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:02:01.243643 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:02:01.243644 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:02:01.243797 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:02:01.243869 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0923 15:02:01.244061 (+ 192us) server_negotiation.cc:299] Negotiation successful
0923 15:02:01.244129 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260923 15:02:01.244318 20684 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:01.226474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:35255 (local address 127.13.75.129:56635)
0923 15:02:01.233229 (+ 6755us) negotiation.cc:107] Waiting for socket to connect
0923 15:02:01.233243 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0923 15:02:01.233695 (+ 452us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:02:01.234179 (+ 484us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:02:01.234180 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:02:01.234461 (+ 281us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:02:01.234514 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:01.234516 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:01.235541 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:01.235542 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:01.236412 (+ 870us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:01.236415 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:01.238023 (+ 1608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:01.238024 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:01.238094 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:01.238561 (+ 467us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:02:01.238567 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:02:01.240504 (+ 1937us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:02:01.242797 (+ 2293us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:02:01.242798 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:02:01.242798 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:02:01.243103 (+ 305us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:02:01.243448 (+ 345us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:02:01.243449 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:02:01.243450 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:02:01.243539 (+ 89us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:02:01.243875 (+ 336us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:02:01.243876 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:02:01.243938 (+ 62us) client_negotiation.cc:769] Sending connection context
0923 15:02:01.244029 (+ 91us) client_negotiation.cc:240] Negotiation successful
0923 15:02:01.244168 (+ 139us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":6730,"thread_start_us":58,"threads_started":1}
I20260923 15:02:01.244712 20678 heartbeater.cc:347] Connected to a master server at 127.13.75.190:35255
I20260923 15:02:01.244771 20678 heartbeater.cc:464] Registering TS with master...
I20260923 15:02:01.244841 20678 heartbeater.cc:511] Master 127.13.75.190:35255 requested a full tablet report, sending...
I20260923 15:02:01.245281 19709 ts_manager.cc:194] Re-registered known tserver with Master: e9ef8b623efa4412ae2bfff361a0a8a8 (127.13.75.129:42411)
I20260923 15:02:01.245690 19709 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.13.75.129@KRBTEST.COM'} at 127.13.75.129:56635
W20260923 15:02:01.323539 20686 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:02:01.323777 20686 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:02:01.323807 20686 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:02:01.325114 20686 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:02:01.325169 20686 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:02:01.325199 20686 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.75.130
I20260923 15:02:01.326853 20686 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.75.148:36053
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/krb5kdc/oryx_127.13.75.130.keytab
--principal=oryx/127.13.75.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.75.130:42895
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.75.130
--webserver_port=44197
--tserver_master_addrs=127.13.75.190:35255,127.13.75.189:44251,127.13.75.188:33585
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20686
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.75.130
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:02:01.327194 20686 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:02:01.327469 20686 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:02:01.333056 20691 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:02:01.333055 20694 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:02:01.333050 20692 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:02:01.333091 20686 server_base.cc:1034] running on GCE node
I20260923 15:02:01.333593 20686 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:02:01.333925 20686 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:02:01.335109 20686 hybrid_clock.cc:648] HybridClock initialized: now 1790175721335092 us; error 32 us; skew 500 ppm
Sep 23 15:02:01 dist-test-slave-cnb0 krb5kdc[19664](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175721, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.75.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:02:01.337961 20686 init.cc:377] Logged in from keytab as oryx/127.13.75.130@KRBTEST.COM (short username oryx)
I20260923 15:02:01.338466 20686 webserver.cc:460] Webserver started at http://127.13.75.130:44197/ using document root <none> and password file <none>
I20260923 15:02:01.338723 20686 fs_manager.cc:359] Metadata directory not provided
I20260923 15:02:01.338814 20686 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:02:01.340700 20686 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:02:01.341395 20701 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:02:01.341701 20686 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:02:01.341821 20686 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/minicluster-data/ts-1/wal
uuid: "a32e8b9f779e4b9288aafcdb2e4a1c7a"
format_stamp: "Formatted at 2026-09-23 15:01:58 on dist-test-slave-cnb0"
server_key: "d76b0efe1e2c1e91807e818dbce81b79"
server_key_iv: "c17c11f711d622f9f0d278b7220ac308"
server_key_version: "encryptionkey@0"
I20260923 15:02:01.342195 20686 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:02:01.364012 20686 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:02:01.366415 20686 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:02:01.366537 20686 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:02:01.366737 20686 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 15:02:01.367146 20686 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 15:02:01.367188 20686 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 15:02:01.367213 20686 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 15:02:01.367261 20686 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 15:02:01.373713 20686 rpc_server.cc:307] RPC server started. Bound to: 127.13.75.130:42895
I20260923 15:02:01.373795 20814 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.75.130:42895 every 8 connection(s)
I20260923 15:02:01.374214 20686 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/minicluster-data/ts-1/data/info.pb
I20260923 15:02:01.380327 13614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKC4soy/build/release/bin/kudu as pid 20686
I20260923 15:02:01.380563 13614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKC4soy/build/release/bin/kudu with pid 20039
Sep 23 15:02:01 dist-test-slave-cnb0 krb5kdc[19664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175721, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.75.130@KRBTEST.COM for oryx/127.13.75.190@KRBTEST.COM
Sep 23 15:02:01 dist-test-slave-cnb0 krb5kdc[19664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175721, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.75.130@KRBTEST.COM for oryx/127.13.75.189@KRBTEST.COM
Sep 23 15:02:01 dist-test-slave-cnb0 krb5kdc[19664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175721, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.75.130@KRBTEST.COM for oryx/127.13.75.188@KRBTEST.COM
I20260923 15:02:01.387579 20245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:01.375628 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.130:48305 (local address 127.13.75.190:35255)
0923 15:02:01.375697 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0923 15:02:01.375700 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:02:01.375748 (+ 48us) server_negotiation.cc:407] Connection header received
0923 15:02:01.376463 (+ 715us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:02:01.376464 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:02:01.376476 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:02:01.376540 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:02:01.377371 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:01.378156 (+ 785us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:01.379808 (+ 1652us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:01.380079 (+ 271us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:01.380155 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:01.383506 (+ 3351us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:02:01.383509 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:02:01.383509 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:02:01.383521 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:02:01.385677 (+ 2156us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:02:01.386236 (+ 559us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:02:01.386238 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:02:01.386238 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:02:01.386301 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:02:01.386615 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:02:01.386616 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:02:01.386616 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:02:01.386780 (+ 164us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:02:01.386827 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0923 15:02:01.387296 (+ 469us) server_negotiation.cc:299] Negotiation successful
0923 15:02:01.387399 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260923 15:02:01.387706 20361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:01.375156 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.130:47691 (local address 127.13.75.189:44251)
0923 15:02:01.375211 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0923 15:02:01.375213 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:02:01.375521 (+ 308us) server_negotiation.cc:407] Connection header received
0923 15:02:01.376196 (+ 675us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:02:01.376198 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:02:01.376212 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:02:01.376263 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:02:01.377363 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:01.377910 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:01.379190 (+ 1280us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:01.379511 (+ 321us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:01.379589 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:01.384124 (+ 4535us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:02:01.384126 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:02:01.384127 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:02:01.384137 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:02:01.386275 (+ 2138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:02:01.386719 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:02:01.386720 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:02:01.386721 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:02:01.386786 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:02:01.387047 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:02:01.387048 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:02:01.387049 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:02:01.387195 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:02:01.387250 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0923 15:02:01.387507 (+ 257us) server_negotiation.cc:299] Negotiation successful
0923 15:02:01.387590 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260923 15:02:01.388357 20820 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:01.375167 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.189:44251 (local address 127.13.75.130:47691)
0923 15:02:01.375402 (+ 235us) negotiation.cc:107] Waiting for socket to connect
0923 15:02:01.375438 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0923 15:02:01.376062 (+ 624us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:02:01.376363 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:02:01.376368 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:02:01.377065 (+ 697us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:02:01.377251 (+ 186us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:01.377257 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:01.378070 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:01.378071 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:01.379077 (+ 1006us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:01.379080 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:01.379608 (+ 528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:01.379608 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:01.379718 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:01.381157 (+ 1439us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:02:01.381163 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:02:01.383987 (+ 2824us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:02:01.386389 (+ 2402us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:02:01.386390 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:02:01.386391 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:02:01.386621 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:02:01.386905 (+ 284us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:02:01.386906 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:02:01.386906 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:02:01.386969 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:02:01.387302 (+ 333us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:02:01.387303 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:02:01.387358 (+ 55us) client_negotiation.cc:769] Sending connection context
0923 15:02:01.387797 (+ 439us) client_negotiation.cc:240] Negotiation successful
0923 15:02:01.388053 (+ 256us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":203,"mutex_wait_us":3,"spinlock_wait_cycles":384,"thread_start_us":141,"threads_started":1}
I20260923 15:02:01.388760 13614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKC4soy/build/release/bin/kudu
/tmp/dist-test-taskKC4soy/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.75.131:35617
--local_ip_for_outbound_sockets=127.13.75.131
--tserver_master_addrs=127.13.75.190:35255
--webserver_port=36865
--webserver_interface=127.13.75.131
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/krb5kdc/oryx_127.13.75.131.keytab
--principal=oryx/127.13.75.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.75.148:36053
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--tserver_master_addrs=127.13.75.190:35255,127.13.75.189:44251
--tserver_master_addrs=127.13.75.190:35255,127.13.75.189:44251,127.13.75.188:33585 with env {KRB5CCNAME=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260923 15:02:01.388972 20682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:01.375151 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.130:59253 (local address 127.13.75.188:33585)
0923 15:02:01.375207 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0923 15:02:01.375210 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:02:01.375534 (+ 324us) server_negotiation.cc:407] Connection header received
0923 15:02:01.376328 (+ 794us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:02:01.376330 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:02:01.376345 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:02:01.376403 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:02:01.377371 (+ 968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:01.378149 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:01.379315 (+ 1166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:01.379693 (+ 378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:01.379751 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:01.385589 (+ 5838us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:02:01.385592 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:02:01.385592 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:02:01.385601 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:02:01.387402 (+ 1801us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:02:01.387769 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:02:01.387770 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:02:01.387771 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:02:01.387830 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:02:01.388183 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:02:01.388184 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:02:01.388184 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:02:01.388346 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:02:01.388541 (+ 195us) server_negotiation.cc:1035] Waiting for connection context
0923 15:02:01.388787 (+ 246us) server_negotiation.cc:299] Negotiation successful
0923 15:02:01.388875 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260923 15:02:01.388368 20821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:01.375515 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:35255 (local address 127.13.75.130:48305)
0923 15:02:01.375695 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0923 15:02:01.375704 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 15:02:01.376064 (+ 360us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:02:01.376550 (+ 486us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:02:01.376551 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:02:01.376965 (+ 414us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:02:01.377253 (+ 288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:01.377259 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:01.378254 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:01.378255 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:01.379231 (+ 976us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:01.379232 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:01.380168 (+ 936us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:01.380169 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:01.380243 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:01.381083 (+ 840us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:02:01.381089 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:02:01.383301 (+ 2212us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:02:01.385780 (+ 2479us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:02:01.385793 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:02:01.385795 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:02:01.386145 (+ 350us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:02:01.386372 (+ 227us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:02:01.386373 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:02:01.386374 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:02:01.386538 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:02:01.386875 (+ 337us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:02:01.386925 (+ 50us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:02:01.387116 (+ 191us) client_negotiation.cc:769] Sending connection context
0923 15:02:01.387843 (+ 727us) client_negotiation.cc:240] Negotiation successful
0923 15:02:01.388142 (+ 299us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":166,"mutex_wait_us":229,"thread_start_us":76,"threads_started":1}
I20260923 15:02:01.389386 20819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:01.375168 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.188:33585 (local address 127.13.75.130:59253)
0923 15:02:01.375378 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0923 15:02:01.375423 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0923 15:02:01.376061 (+ 638us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:02:01.376416 (+ 355us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:02:01.376417 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:02:01.376964 (+ 547us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:02:01.377253 (+ 289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:01.377259 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:01.378254 (+ 995us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:01.378255 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:01.379227 (+ 972us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:01.379230 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:01.379768 (+ 538us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:01.379769 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:01.379823 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:01.380578 (+ 755us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:02:01.380589 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:02:01.385467 (+ 4878us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:02:01.387485 (+ 2018us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:02:01.387486 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:02:01.387487 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:02:01.387681 (+ 194us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:02:01.387976 (+ 295us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:02:01.387977 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:02:01.387977 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:02:01.388058 (+ 81us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:02:01.388618 (+ 560us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:02:01.388618 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:02:01.388663 (+ 45us) client_negotiation.cc:769] Sending connection context
0923 15:02:01.388759 (+ 96us) client_negotiation.cc:240] Negotiation successful
0923 15:02:01.388866 (+ 107us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":167,"mutex_wait_us":38,"thread_start_us":105,"threads_started":1}
I20260923 15:02:01.389952 20815 heartbeater.cc:347] Connected to a master server at 127.13.75.190:35255
I20260923 15:02:01.389977 20816 heartbeater.cc:347] Connected to a master server at 127.13.75.188:33585
I20260923 15:02:01.390040 20815 heartbeater.cc:464] Registering TS with master...
I20260923 15:02:01.390048 20816 heartbeater.cc:464] Registering TS with master...
I20260923 15:02:01.390290 20816 heartbeater.cc:511] Master 127.13.75.188:33585 requested a full tablet report, sending...
I20260923 15:02:01.390290 20815 heartbeater.cc:511] Master 127.13.75.190:35255 requested a full tablet report, sending...
I20260923 15:02:01.390590 20817 heartbeater.cc:347] Connected to a master server at 127.13.75.189:44251
I20260923 15:02:01.390630 20817 heartbeater.cc:464] Registering TS with master...
I20260923 15:02:01.390693 20817 heartbeater.cc:511] Master 127.13.75.189:44251 requested a full tablet report, sending...
I20260923 15:02:01.390714 19709 ts_manager.cc:194] Re-registered known tserver with Master: a32e8b9f779e4b9288aafcdb2e4a1c7a (127.13.75.130:42895)
I20260923 15:02:01.390772 20499 ts_manager.cc:194] Registered new tserver with Master: a32e8b9f779e4b9288aafcdb2e4a1c7a (127.13.75.130:42895)
I20260923 15:02:01.391291 19709 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.13.75.130@KRBTEST.COM'} at 127.13.75.130:48305
I20260923 15:02:01.391654 20310 ts_manager.cc:194] Registered new tserver with Master: a32e8b9f779e4b9288aafcdb2e4a1c7a (127.13.75.130:42895)
W20260923 15:02:01.476753 20822 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:02:01.477018 20822 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:02:01.477062 20822 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:02:01.478327 20822 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:02:01.478400 20822 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:02:01.478447 20822 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.75.131
I20260923 15:02:01.480126 20822 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.75.148:36053
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/krb5kdc/oryx_127.13.75.131.keytab
--principal=oryx/127.13.75.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.75.131:35617
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.75.131
--webserver_port=36865
--tserver_master_addrs=127.13.75.190:35255,127.13.75.189:44251,127.13.75.188:33585
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.20822
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.75.131
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:02:01.480427 20822 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:02:01.480809 20822 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:02:01.486181 20827 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:02:01.486207 20828 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:02:01.486176 20830 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:02:01.486619 20822 server_base.cc:1034] running on GCE node
I20260923 15:02:01.486851 20822 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:02:01.487159 20822 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:02:01.488318 20822 hybrid_clock.cc:648] HybridClock initialized: now 1790175721488294 us; error 38 us; skew 500 ppm
Sep 23 15:02:01 dist-test-slave-cnb0 krb5kdc[19664](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175721, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.75.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:02:01.491200 20822 init.cc:377] Logged in from keytab as oryx/127.13.75.131@KRBTEST.COM (short username oryx)
I20260923 15:02:01.491714 20822 webserver.cc:460] Webserver started at http://127.13.75.131:36865/ using document root <none> and password file <none>
I20260923 15:02:01.491963 20822 fs_manager.cc:359] Metadata directory not provided
I20260923 15:02:01.492051 20822 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:02:01.493655 20822 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:02:01.494572 20837 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:02:01.494800 20822 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:02:01.495007 20822 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/minicluster-data/ts-2/wal
uuid: "430f9682c1fe49f7b9fc238400127887"
format_stamp: "Formatted at 2026-09-23 15:01:59 on dist-test-slave-cnb0"
server_key: "938d7bf162a733df869127699b1592af"
server_key_iv: "e29dabf2097eefbf2db97bbbe59360a0"
server_key_version: "encryptionkey@0"
I20260923 15:02:01.495462 20822 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:02:01.526298 20822 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:02:01.528788 20822 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:02:01.528914 20822 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:02:01.529198 20822 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 15:02:01.529601 20822 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 15:02:01.529639 20822 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 15:02:01.529661 20822 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 15:02:01.529675 20822 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 15:02:01.536723 20822 rpc_server.cc:307] RPC server started. Bound to: 127.13.75.131:35617
I20260923 15:02:01.536763 20950 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.75.131:35617 every 8 connection(s)
I20260923 15:02:01.537201 20822 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/minicluster-data/ts-2/data/info.pb
Sep 23 15:02:01 dist-test-slave-cnb0 krb5kdc[19664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175721, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.75.131@KRBTEST.COM for oryx/127.13.75.188@KRBTEST.COM
I20260923 15:02:01.545336 13614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKC4soy/build/release/bin/kudu as pid 20822
Sep 23 15:02:01 dist-test-slave-cnb0 krb5kdc[19664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175721, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.75.131@KRBTEST.COM for oryx/127.13.75.190@KRBTEST.COM
Sep 23 15:02:01 dist-test-slave-cnb0 krb5kdc[19664](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175721, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.75.131@KRBTEST.COM for oryx/127.13.75.189@KRBTEST.COM
I20260923 15:02:01.548131 20269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:01.544104 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.188:33585 (local address 127.0.0.1:41044)
0923 15:02:01.544162 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0923 15:02:01.544173 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 15:02:01.544216 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:02:01.544481 (+ 265us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:02:01.544482 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:02:01.544484 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:02:01.544529 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:01.544530 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:01.545683 (+ 1153us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:01.545685 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:01.546790 (+ 1105us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:01.546794 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:01.547839 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:01.547840 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:01.547912 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:01.547916 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 15:02:01.547970 (+ 54us) client_negotiation.cc:240] Negotiation successful
0923 15:02:01.548028 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":40}
I20260923 15:02:01.548394 20958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:01.544168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41044 (local address 127.13.75.188:33585)
0923 15:02:01.544299 (+ 131us) server_negotiation.cc:206] Beginning negotiation
0923 15:02:01.544300 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 15:02:01.544309 (+ 9us) server_negotiation.cc:407] Connection header received
0923 15:02:01.544333 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:02:01.544333 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:02:01.544358 (+ 25us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:02:01.544471 (+ 113us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:02:01.544625 (+ 154us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:01.545499 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:01.546923 (+ 1424us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:01.547619 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:01.547692 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:01.547782 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0923 15:02:01.548241 (+ 459us) server_negotiation.cc:299] Negotiation successful
0923 15:02:01.548297 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":78,"threads_started":1}
I20260923 15:02:01.549542 20509 raft_consensus.cc:1203] T 00000000000000000000000000000000 P 28aa43d448d94ccdb6d8776e93de4fc9 [term 1 LEARNER]: Deduplicated request from leader. Original: 1.6->[1.7-1.7] Dedup: 1.7->[]
I20260923 15:02:01.549767 20541 sys_catalog.cc:455] T 00000000000000000000000000000000 P 28aa43d448d94ccdb6d8776e93de4fc9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 176378cdc0584ce69eca703063507558. Latest consensus state: current_term: 1 leader_uuid: "176378cdc0584ce69eca703063507558" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "176378cdc0584ce69eca703063507558" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 35255 } } peers { permanent_uuid: "b8c37de502e542aab23126e30c0e3176" member_type: VOTER last_known_addr { host: "127.13.75.189" port: 44251 } attrs { promote: false } } peers { permanent_uuid: "28aa43d448d94ccdb6d8776e93de4fc9" member_type: NON_VOTER last_known_addr { host: "127.13.75.188" port: 33585 } attrs { promote: true } } }
I20260923 15:02:01.549883 20541 sys_catalog.cc:458] T 00000000000000000000000000000000 P 28aa43d448d94ccdb6d8776e93de4fc9 [sys.catalog]: This master's current role is: LEARNER
I20260923 15:02:01.550680 20361 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:01.538060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.131:51059 (local address 127.13.75.189:44251)
0923 15:02:01.538113 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0923 15:02:01.538116 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:02:01.538448 (+ 332us) server_negotiation.cc:407] Connection header received
0923 15:02:01.539623 (+ 1175us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:02:01.539625 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:02:01.539641 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:02:01.539692 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:02:01.541482 (+ 1790us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:01.542224 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:01.543263 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:01.543483 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:01.543567 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:01.547004 (+ 3437us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:02:01.547006 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:02:01.547006 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:02:01.547014 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:02:01.549034 (+ 2020us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:02:01.549491 (+ 457us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:02:01.549492 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:02:01.549493 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:02:01.549564 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:02:01.549886 (+ 322us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:02:01.549887 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:02:01.549888 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:02:01.550041 (+ 153us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:02:01.550104 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0923 15:02:01.550479 (+ 375us) server_negotiation.cc:299] Negotiation successful
0923 15:02:01.550576 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260923 15:02:01.550945 20955 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:01.538074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.188:33585 (local address 127.13.75.131:57585)
0923 15:02:01.538276 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0923 15:02:01.538321 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0923 15:02:01.539572 (+ 1251us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:02:01.539848 (+ 276us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:02:01.539849 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:02:01.540360 (+ 511us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:02:01.540586 (+ 226us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:01.540593 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:01.541446 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:01.541447 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:01.542450 (+ 1003us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:01.542452 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:01.542923 (+ 471us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:01.542924 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:01.543067 (+ 143us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:01.543644 (+ 577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:02:01.543657 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:02:01.546858 (+ 3201us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:02:01.549368 (+ 2510us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:02:01.549369 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:02:01.549369 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:02:01.549593 (+ 224us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:02:01.549817 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:02:01.549818 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:02:01.549818 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:02:01.549876 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:02:01.550195 (+ 319us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:02:01.550195 (+ 0us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:02:01.550294 (+ 99us) client_negotiation.cc:769] Sending connection context
0923 15:02:01.550407 (+ 113us) client_negotiation.cc:240] Negotiation successful
0923 15:02:01.550650 (+ 243us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"mutex_wait_us":56,"thread_start_us":118,"threads_started":1}
I20260923 15:02:01.551399 20956 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:01.538074 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.189:44251 (local address 127.13.75.131:51059)
0923 15:02:01.538277 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0923 15:02:01.538321 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0923 15:02:01.539493 (+ 1172us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:02:01.539835 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:02:01.539840 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:02:01.540361 (+ 521us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:02:01.540585 (+ 224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:01.540593 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:01.542320 (+ 1727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:01.542321 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:01.543172 (+ 851us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:01.543175 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:01.543591 (+ 416us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:01.543592 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:01.543672 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:01.544087 (+ 415us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:02:01.544092 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:02:01.546875 (+ 2783us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:02:01.549127 (+ 2252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:02:01.549138 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:02:01.549140 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:02:01.549404 (+ 264us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:02:01.549673 (+ 269us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:02:01.549674 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:02:01.549674 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:02:01.549819 (+ 145us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:02:01.550110 (+ 291us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:02:01.550117 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:02:01.550294 (+ 177us) client_negotiation.cc:769] Sending connection context
0923 15:02:01.550426 (+ 132us) client_negotiation.cc:240] Negotiation successful
0923 15:02:01.550700 (+ 274us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":169,"mutex_wait_us":154,"thread_start_us":134,"threads_started":1}
I20260923 15:02:01.551437 20682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:01.538067 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.131:57585 (local address 127.13.75.188:33585)
0923 15:02:01.538115 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0923 15:02:01.538117 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:02:01.538425 (+ 308us) server_negotiation.cc:407] Connection header received
0923 15:02:01.539683 (+ 1258us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:02:01.539684 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:02:01.539699 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:02:01.539731 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:02:01.540759 (+ 1028us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:01.541360 (+ 601us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:01.542550 (+ 1190us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:01.542814 (+ 264us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:01.542924 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:01.546982 (+ 4058us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:02:01.546983 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:02:01.546984 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:02:01.546992 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:02:01.549268 (+ 2276us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:02:01.549681 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:02:01.549682 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:02:01.549682 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:02:01.549763 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:02:01.549950 (+ 187us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:02:01.549951 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:02:01.549951 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:02:01.550102 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:02:01.550163 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0923 15:02:01.551255 (+ 1092us) server_negotiation.cc:299] Negotiation successful
0923 15:02:01.551345 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260923 15:02:01.552060 20952 heartbeater.cc:347] Connected to a master server at 127.13.75.188:33585
I20260923 15:02:01.552161 20952 heartbeater.cc:464] Registering TS with master...
I20260923 15:02:01.552212 20953 heartbeater.cc:347] Connected to a master server at 127.13.75.189:44251
I20260923 15:02:01.552244 20953 heartbeater.cc:464] Registering TS with master...
I20260923 15:02:01.552366 20952 heartbeater.cc:511] Master 127.13.75.188:33585 requested a full tablet report, sending...
I20260923 15:02:01.552380 20957 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:01.538408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:35255 (local address 127.13.75.131:60895)
0923 15:02:01.538546 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0923 15:02:01.538556 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 15:02:01.539548 (+ 992us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:02:01.539890 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:02:01.539890 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:02:01.540359 (+ 469us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:02:01.540586 (+ 227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:01.540593 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:01.541440 (+ 847us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:01.541441 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:01.542425 (+ 984us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:01.542427 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:01.542991 (+ 564us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:01.542991 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:01.543068 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:01.543764 (+ 696us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:02:01.543769 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:02:01.546346 (+ 2577us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:02:01.551220 (+ 4874us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:02:01.551221 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:02:01.551222 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:02:01.551445 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:02:01.551673 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:02:01.551674 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:02:01.551675 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:02:01.551737 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:02:01.552030 (+ 293us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:02:01.552031 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:02:01.552087 (+ 56us) client_negotiation.cc:769] Sending connection context
0923 15:02:01.552150 (+ 63us) client_negotiation.cc:240] Negotiation successful
0923 15:02:01.552251 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"mutex_wait_us":31,"thread_start_us":55,"threads_started":1}
I20260923 15:02:01.552366 20953 heartbeater.cc:511] Master 127.13.75.189:44251 requested a full tablet report, sending...
I20260923 15:02:01.552623 20245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:01.538586 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.131:60895 (local address 127.13.75.190:35255)
0923 15:02:01.538631 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0923 15:02:01.538633 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:02:01.538641 (+ 8us) server_negotiation.cc:407] Connection header received
0923 15:02:01.539757 (+ 1116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:02:01.539759 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:02:01.539772 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:02:01.539821 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:02:01.540703 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:01.541326 (+ 623us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:01.542528 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:01.542835 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:01.542878 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:01.546447 (+ 3569us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:02:01.546449 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:02:01.546449 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:02:01.546456 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:02:01.551133 (+ 4677us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:02:01.551532 (+ 399us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:02:01.551534 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:02:01.551535 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:02:01.551596 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:02:01.551806 (+ 210us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:02:01.551806 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:02:01.551807 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:02:01.551936 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:02:01.552002 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0923 15:02:01.552454 (+ 452us) server_negotiation.cc:299] Negotiation successful
0923 15:02:01.552537 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260923 15:02:01.552845 20310 ts_manager.cc:194] Registered new tserver with Master: 430f9682c1fe49f7b9fc238400127887 (127.13.75.131:35617)
I20260923 15:02:01.553233 20499 ts_manager.cc:194] Registered new tserver with Master: 430f9682c1fe49f7b9fc238400127887 (127.13.75.131:35617)
I20260923 15:02:01.553290 20951 heartbeater.cc:347] Connected to a master server at 127.13.75.190:35255
I20260923 15:02:01.553332 20951 heartbeater.cc:464] Registering TS with master...
I20260923 15:02:01.553398 20951 heartbeater.cc:511] Master 127.13.75.190:35255 requested a full tablet report, sending...
I20260923 15:02:01.553676 19709 ts_manager.cc:194] Re-registered known tserver with Master: 430f9682c1fe49f7b9fc238400127887 (127.13.75.131:35617)
I20260923 15:02:01.554157 19709 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.13.75.131@KRBTEST.COM'} at 127.13.75.131:60895
I20260923 15:02:01.560181 20245 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:01.547633 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50810 (local address 127.13.75.190:35255)
0923 15:02:01.552695 (+ 5062us) server_negotiation.cc:206] Beginning negotiation
0923 15:02:01.552697 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:02:01.552719 (+ 22us) server_negotiation.cc:407] Connection header received
0923 15:02:01.552737 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:02:01.552737 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:02:01.552746 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:02:01.552805 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:02:01.553708 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:01.554394 (+ 686us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:01.555435 (+ 1041us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:01.555632 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:01.555719 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:01.557158 (+ 1439us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:02:01.557159 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:02:01.557160 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:02:01.557167 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:02:01.558870 (+ 1703us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:02:01.559302 (+ 432us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:02:01.559303 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:02:01.559303 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:02:01.559361 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:02:01.559584 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:02:01.559585 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:02:01.559585 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:02:01.559736 (+ 151us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:02:01.559833 (+ 97us) server_negotiation.cc:1035] Waiting for connection context
0923 15:02:01.559916 (+ 83us) server_negotiation.cc:299] Negotiation successful
0923 15:02:01.559994 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":5047,"thread_start_us":86,"threads_started":1}
I20260923 15:02:01.560577 20966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:01.548060 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39182 (local address 127.13.75.189:44251)
0923 15:02:01.550241 (+ 2181us) server_negotiation.cc:206] Beginning negotiation
0923 15:02:01.550243 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:02:01.550973 (+ 730us) server_negotiation.cc:407] Connection header received
0923 15:02:01.551115 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:02:01.551116 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:02:01.551130 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:02:01.551187 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:02:01.552019 (+ 832us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:01.552761 (+ 742us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:01.553959 (+ 1198us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:01.554342 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:01.554409 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:01.557175 (+ 2766us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:02:01.557177 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:02:01.557177 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:02:01.557185 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:02:01.559094 (+ 1909us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:02:01.559476 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:02:01.559477 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:02:01.559478 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:02:01.559531 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:02:01.559751 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:02:01.559753 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:02:01.559753 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:02:01.559899 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:02:01.560004 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0923 15:02:01.560366 (+ 362us) server_negotiation.cc:299] Negotiation successful
0923 15:02:01.560450 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2162,"thread_start_us":59,"threads_started":1}
I20260923 15:02:01.561546 20958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:01.547691 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41048 (local address 127.13.75.188:33585)
0923 15:02:01.548482 (+ 791us) server_negotiation.cc:206] Beginning negotiation
0923 15:02:01.548484 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:02:01.552936 (+ 4452us) server_negotiation.cc:407] Connection header received
0923 15:02:01.554052 (+ 1116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:02:01.554053 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:02:01.554063 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:02:01.554090 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:02:01.554796 (+ 706us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:01.555509 (+ 713us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:01.556564 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:01.556897 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:01.556941 (+ 44us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:01.558368 (+ 1427us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:02:01.558370 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:02:01.558370 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:02:01.558378 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:02:01.560214 (+ 1836us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:02:01.560565 (+ 351us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:02:01.560566 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:02:01.560567 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:02:01.560634 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:02:01.560948 (+ 314us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:02:01.560949 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:02:01.560949 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:02:01.561106 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:02:01.561207 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0923 15:02:01.561358 (+ 151us) server_negotiation.cc:299] Negotiation successful
0923 15:02:01.561450 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":774,"thread_start_us":51,"threads_started":1}
I20260923 15:02:01.561859 19709 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50810:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 15:02:01.563786 19709 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 15:02:01.564630 20462 sys_catalog.cc:455] T 00000000000000000000000000000000 P 176378cdc0584ce69eca703063507558 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "176378cdc0584ce69eca703063507558" committed_config { opid_index: 7 OBSOLETE_local: true peers { permanent_uuid: "176378cdc0584ce69eca703063507558" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 35255 } } peers { permanent_uuid: "b8c37de502e542aab23126e30c0e3176" member_type: VOTER last_known_addr { host: "127.13.75.189" port: 44251 } attrs { promote: false } } peers { permanent_uuid: "28aa43d448d94ccdb6d8776e93de4fc9" member_type: NON_VOTER last_known_addr { host: "127.13.75.188" port: 33585 } attrs { promote: true } } }
I20260923 15:02:01.564759 20462 sys_catalog.cc:458] T 00000000000000000000000000000000 P 176378cdc0584ce69eca703063507558 [sys.catalog]: This master's current role is: LEADER
I20260923 15:02:01.565897 20459 raft_consensus.cc:1050] T 00000000000000000000000000000000 P 176378cdc0584ce69eca703063507558: attempting to promote NON_VOTER 28aa43d448d94ccdb6d8776e93de4fc9 to VOTER
I20260923 15:02:01.566027 20459 consensus_queue.cc:237] T 00000000000000000000000000000000 P 176378cdc0584ce69eca703063507558 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "176378cdc0584ce69eca703063507558" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 35255 } } peers { permanent_uuid: "b8c37de502e542aab23126e30c0e3176" member_type: VOTER last_known_addr { host: "127.13.75.189" port: 44251 } attrs { promote: false } } peers { permanent_uuid: "28aa43d448d94ccdb6d8776e93de4fc9" member_type: VOTER last_known_addr { host: "127.13.75.188" port: 33585 } attrs { promote: false } }
I20260923 15:02:01.566401 20509 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 28aa43d448d94ccdb6d8776e93de4fc9 [term 1 LEARNER]: Refusing update from remote peer 176378cdc0584ce69eca703063507558: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260923 15:02:01.566499 20320 raft_consensus.cc:1261] T 00000000000000000000000000000000 P b8c37de502e542aab23126e30c0e3176 [term 1 FOLLOWER]: Refusing update from remote peer 176378cdc0584ce69eca703063507558: Log matching property violated. Preceding OpId in replica: term: 1 index: 8. Preceding OpId from leader: term: 1 index: 9. (index mismatch)
I20260923 15:02:01.566650 20462 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 176378cdc0584ce69eca703063507558 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28aa43d448d94ccdb6d8776e93de4fc9" member_type: VOTER last_known_addr { host: "127.13.75.188" port: 33585 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260923 15:02:01.566920 20462 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 176378cdc0584ce69eca703063507558 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b8c37de502e542aab23126e30c0e3176" member_type: VOTER last_known_addr { host: "127.13.75.189" port: 44251 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 9, Last known committed idx: 8, Time since last communication: 0.000s
I20260923 15:02:01.567487 20271 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 176378cdc0584ce69eca703063507558 [term 1 LEADER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 28aa43d448d94ccdb6d8776e93de4fc9 (127.13.75.188) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "176378cdc0584ce69eca703063507558" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 35255 } } peers { permanent_uuid: "b8c37de502e542aab23126e30c0e3176" member_type: VOTER last_known_addr { host: "127.13.75.189" port: 44251 } attrs { promote: false } } peers { permanent_uuid: "28aa43d448d94ccdb6d8776e93de4fc9" member_type: VOTER last_known_addr { host: "127.13.75.188" port: 33585 } attrs { promote: false } } }
I20260923 15:02:01.568310 20463 sys_catalog.cc:455] T 00000000000000000000000000000000 P 176378cdc0584ce69eca703063507558 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "176378cdc0584ce69eca703063507558" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "176378cdc0584ce69eca703063507558" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 35255 } } peers { permanent_uuid: "b8c37de502e542aab23126e30c0e3176" member_type: VOTER last_known_addr { host: "127.13.75.189" port: 44251 } attrs { promote: false } } peers { permanent_uuid: "28aa43d448d94ccdb6d8776e93de4fc9" member_type: VOTER last_known_addr { host: "127.13.75.188" port: 33585 } attrs { promote: false } } }
I20260923 15:02:01.568393 20463 sys_catalog.cc:458] T 00000000000000000000000000000000 P 176378cdc0584ce69eca703063507558 [sys.catalog]: This master's current role is: LEADER
I20260923 15:02:01.568287 20271 sys_catalog.cc:455] T 00000000000000000000000000000000 P 176378cdc0584ce69eca703063507558 [sys.catalog]: SysCatalogTable state changed. Reason: Config change replication complete. Latest consensus state: current_term: 1 leader_uuid: "176378cdc0584ce69eca703063507558" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "176378cdc0584ce69eca703063507558" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 35255 } } peers { permanent_uuid: "b8c37de502e542aab23126e30c0e3176" member_type: VOTER last_known_addr { host: "127.13.75.189" port: 44251 } attrs { promote: false } } peers { permanent_uuid: "28aa43d448d94ccdb6d8776e93de4fc9" member_type: VOTER last_known_addr { host: "127.13.75.188" port: 33585 } attrs { promote: false } } }
I20260923 15:02:01.568470 20271 sys_catalog.cc:458] T 00000000000000000000000000000000 P 176378cdc0584ce69eca703063507558 [sys.catalog]: This master's current role is: LEADER
I20260923 15:02:01.568650 20271 sys_catalog.cc:455] T 00000000000000000000000000000000 P 176378cdc0584ce69eca703063507558 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "176378cdc0584ce69eca703063507558" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "176378cdc0584ce69eca703063507558" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 35255 } } peers { permanent_uuid: "b8c37de502e542aab23126e30c0e3176" member_type: VOTER last_known_addr { host: "127.13.75.189" port: 44251 } attrs { promote: false } } peers { permanent_uuid: "28aa43d448d94ccdb6d8776e93de4fc9" member_type: VOTER last_known_addr { host: "127.13.75.188" port: 33585 } attrs { promote: false } } }
I20260923 15:02:01.568713 20271 sys_catalog.cc:458] T 00000000000000000000000000000000 P 176378cdc0584ce69eca703063507558 [sys.catalog]: This master's current role is: LEADER
I20260923 15:02:01.568630 20509 raft_consensus.cc:2932] T 00000000000000000000000000000000 P 28aa43d448d94ccdb6d8776e93de4fc9 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 28aa43d448d94ccdb6d8776e93de4fc9 (127.13.75.188) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "176378cdc0584ce69eca703063507558" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 35255 } } peers { permanent_uuid: "b8c37de502e542aab23126e30c0e3176" member_type: VOTER last_known_addr { host: "127.13.75.189" port: 44251 } attrs { promote: false } } peers { permanent_uuid: "28aa43d448d94ccdb6d8776e93de4fc9" member_type: VOTER last_known_addr { host: "127.13.75.188" port: 33585 } attrs { promote: false } } }
I20260923 15:02:01.568908 20320 raft_consensus.cc:2932] T 00000000000000000000000000000000 P b8c37de502e542aab23126e30c0e3176 [term 1 FOLLOWER]: Committing config change with OpId 1.9: config changed from index 7 to 9, 28aa43d448d94ccdb6d8776e93de4fc9 (127.13.75.188) changed from NON_VOTER to VOTER. New config: { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "176378cdc0584ce69eca703063507558" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 35255 } } peers { permanent_uuid: "b8c37de502e542aab23126e30c0e3176" member_type: VOTER last_known_addr { host: "127.13.75.189" port: 44251 } attrs { promote: false } } peers { permanent_uuid: "28aa43d448d94ccdb6d8776e93de4fc9" member_type: VOTER last_known_addr { host: "127.13.75.188" port: 33585 } attrs { promote: false } } }
I20260923 15:02:01.569804 20541 sys_catalog.cc:455] T 00000000000000000000000000000000 P 28aa43d448d94ccdb6d8776e93de4fc9 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "176378cdc0584ce69eca703063507558" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "176378cdc0584ce69eca703063507558" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 35255 } } peers { permanent_uuid: "b8c37de502e542aab23126e30c0e3176" member_type: VOTER last_known_addr { host: "127.13.75.189" port: 44251 } attrs { promote: false } } peers { permanent_uuid: "28aa43d448d94ccdb6d8776e93de4fc9" member_type: VOTER last_known_addr { host: "127.13.75.188" port: 33585 } attrs { promote: false } } }
I20260923 15:02:01.569898 20541 sys_catalog.cc:458] T 00000000000000000000000000000000 P 28aa43d448d94ccdb6d8776e93de4fc9 [sys.catalog]: This master's current role is: FOLLOWER
I20260923 15:02:01.570317 20461 sys_catalog.cc:455] T 00000000000000000000000000000000 P b8c37de502e542aab23126e30c0e3176 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "176378cdc0584ce69eca703063507558" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "176378cdc0584ce69eca703063507558" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 35255 } } peers { permanent_uuid: "b8c37de502e542aab23126e30c0e3176" member_type: VOTER last_known_addr { host: "127.13.75.189" port: 44251 } attrs { promote: false } } peers { permanent_uuid: "28aa43d448d94ccdb6d8776e93de4fc9" member_type: VOTER last_known_addr { host: "127.13.75.188" port: 33585 } attrs { promote: false } } }
I20260923 15:02:01.570428 20461 sys_catalog.cc:458] T 00000000000000000000000000000000 P b8c37de502e542aab23126e30c0e3176 [sys.catalog]: This master's current role is: FOLLOWER
I20260923 15:02:01.574451 20979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:01.569935 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60130 (local address 127.13.75.130:42895)
0923 15:02:01.570780 (+ 845us) server_negotiation.cc:206] Beginning negotiation
0923 15:02:01.570785 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 15:02:01.571102 (+ 317us) server_negotiation.cc:407] Connection header received
0923 15:02:01.571148 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:02:01.571151 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:02:01.571167 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:02:01.571234 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:02:01.571335 (+ 101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:01.572471 (+ 1136us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:01.573545 (+ 1074us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:01.574055 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:01.574156 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:01.574240 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0923 15:02:01.574311 (+ 71us) server_negotiation.cc:299] Negotiation successful
0923 15:02:01.574349 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":807,"thread_start_us":84,"threads_started":1}
I20260923 15:02:01.574487 20269 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:01.569426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.129:42411 (local address 127.0.0.1:51176)
0923 15:02:01.569757 (+ 331us) negotiation.cc:107] Waiting for socket to connect
0923 15:02:01.569770 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0923 15:02:01.569814 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:02:01.570541 (+ 727us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:02:01.570542 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:02:01.570544 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:02:01.570600 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:01.570602 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:01.572086 (+ 1484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:01.572087 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:01.573254 (+ 1167us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:01.573256 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:01.574213 (+ 957us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:01.574214 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:01.574288 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:01.574292 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 15:02:01.574322 (+ 30us) client_negotiation.cc:240] Negotiation successful
0923 15:02:01.574366 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":311,"mutex_wait_us":32}
I20260923 15:02:01.574683 20977 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:01.569459 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.131:35617 (local address 127.0.0.1:55026)
0923 15:02:01.569998 (+ 539us) negotiation.cc:107] Waiting for socket to connect
0923 15:02:01.570008 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 15:02:01.570056 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:02:01.571074 (+ 1018us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:02:01.571076 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:02:01.571078 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:02:01.571127 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:01.571129 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:01.572022 (+ 893us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:01.572023 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:01.573253 (+ 1230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:01.573256 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:01.574294 (+ 1038us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:01.574295 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:01.574351 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:01.574354 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 15:02:01.574399 (+ 45us) client_negotiation.cc:240] Negotiation successful
0923 15:02:01.574438 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":518,"mutex_wait_us":40,"thread_start_us":87,"threads_started":1}
I20260923 15:02:01.574877 20978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:01.569762 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55026 (local address 127.13.75.131:35617)
0923 15:02:01.570874 (+ 1112us) server_negotiation.cc:206] Beginning negotiation
0923 15:02:01.570904 (+ 30us) server_negotiation.cc:399] Waiting for connection header
0923 15:02:01.570914 (+ 10us) server_negotiation.cc:407] Connection header received
0923 15:02:01.570961 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:02:01.570963 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:02:01.570981 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:02:01.571089 (+ 108us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:02:01.571175 (+ 86us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:01.571914 (+ 739us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:01.573362 (+ 1448us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:01.574107 (+ 745us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:01.574183 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:01.574263 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0923 15:02:01.574532 (+ 269us) server_negotiation.cc:299] Negotiation successful
0923 15:02:01.574595 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1076,"thread_start_us":80,"threads_started":1}
I20260923 15:02:01.574504 20976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:01.569555 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.130:42895 (local address 127.0.0.1:60130)
0923 15:02:01.570953 (+ 1398us) negotiation.cc:107] Waiting for socket to connect
0923 15:02:01.570971 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0923 15:02:01.571025 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:02:01.571239 (+ 214us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:02:01.571240 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:02:01.571242 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:02:01.571280 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:01.571281 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:01.572556 (+ 1275us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:01.572558 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:01.573460 (+ 902us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:01.573462 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:01.574162 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:01.574163 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:01.574224 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:01.574229 (+ 5us) client_negotiation.cc:769] Sending connection context
0923 15:02:01.574295 (+ 66us) client_negotiation.cc:240] Negotiation successful
0923 15:02:01.574341 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1382,"thread_start_us":81,"threads_started":1}
I20260923 15:02:01.575449 20980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:01.570134 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51176 (local address 127.13.75.129:42411)
0923 15:02:01.570326 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0923 15:02:01.570331 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 15:02:01.570342 (+ 11us) server_negotiation.cc:407] Connection header received
0923 15:02:01.570395 (+ 53us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:02:01.570397 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:02:01.570416 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:02:01.570549 (+ 133us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:02:01.570719 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:01.571981 (+ 1262us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:01.573362 (+ 1381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:01.574112 (+ 750us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:01.574848 (+ 736us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:01.574972 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0923 15:02:01.575312 (+ 340us) server_negotiation.cc:299] Negotiation successful
0923 15:02:01.575369 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":95,"threads_started":1}
I20260923 15:02:01.575749 20885 tablet_service.cc:1467] Processing CreateTablet for tablet bf6189755d9f4d72966ef3aef88aa391 (DEFAULT_TABLE table=test-table [id=22366170de2a42f7b1654672c9d2566c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 15:02:01.575866 20749 tablet_service.cc:1467] Processing CreateTablet for tablet bf6189755d9f4d72966ef3aef88aa391 (DEFAULT_TABLE table=test-table [id=22366170de2a42f7b1654672c9d2566c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 15:02:01.576141 20749 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bf6189755d9f4d72966ef3aef88aa391. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:02:01.576156 20885 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bf6189755d9f4d72966ef3aef88aa391. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:02:01.576067 20612 tablet_service.cc:1467] Processing CreateTablet for tablet bf6189755d9f4d72966ef3aef88aa391 (DEFAULT_TABLE table=test-table [id=22366170de2a42f7b1654672c9d2566c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 15:02:01.576373 20612 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bf6189755d9f4d72966ef3aef88aa391. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:02:01.578810 20981 tablet_bootstrap.cc:492] T bf6189755d9f4d72966ef3aef88aa391 P a32e8b9f779e4b9288aafcdb2e4a1c7a: Bootstrap starting.
I20260923 15:02:01.579527 20981 tablet_bootstrap.cc:654] T bf6189755d9f4d72966ef3aef88aa391 P a32e8b9f779e4b9288aafcdb2e4a1c7a: Neither blocks nor log segments found. Creating new log.
I20260923 15:02:01.579814 20983 tablet_bootstrap.cc:492] T bf6189755d9f4d72966ef3aef88aa391 P e9ef8b623efa4412ae2bfff361a0a8a8: Bootstrap starting.
I20260923 15:02:01.579910 20981 log.cc:824] T bf6189755d9f4d72966ef3aef88aa391 P a32e8b9f779e4b9288aafcdb2e4a1c7a: Log is configured to *not* fsync() on all Append() calls
I20260923 15:02:01.580485 20983 tablet_bootstrap.cc:654] T bf6189755d9f4d72966ef3aef88aa391 P e9ef8b623efa4412ae2bfff361a0a8a8: Neither blocks nor log segments found. Creating new log.
I20260923 15:02:01.580646 20982 tablet_bootstrap.cc:492] T bf6189755d9f4d72966ef3aef88aa391 P 430f9682c1fe49f7b9fc238400127887: Bootstrap starting.
I20260923 15:02:01.580878 20983 log.cc:824] T bf6189755d9f4d72966ef3aef88aa391 P e9ef8b623efa4412ae2bfff361a0a8a8: Log is configured to *not* fsync() on all Append() calls
I20260923 15:02:01.580909 20981 tablet_bootstrap.cc:492] T bf6189755d9f4d72966ef3aef88aa391 P a32e8b9f779e4b9288aafcdb2e4a1c7a: No bootstrap required, opened a new log
I20260923 15:02:01.580974 20981 ts_tablet_manager.cc:1397] T bf6189755d9f4d72966ef3aef88aa391 P a32e8b9f779e4b9288aafcdb2e4a1c7a: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 15:02:01.581369 20982 tablet_bootstrap.cc:654] T bf6189755d9f4d72966ef3aef88aa391 P 430f9682c1fe49f7b9fc238400127887: Neither blocks nor log segments found. Creating new log.
I20260923 15:02:01.581676 20982 log.cc:824] T bf6189755d9f4d72966ef3aef88aa391 P 430f9682c1fe49f7b9fc238400127887: Log is configured to *not* fsync() on all Append() calls
I20260923 15:02:01.581708 20983 tablet_bootstrap.cc:492] T bf6189755d9f4d72966ef3aef88aa391 P e9ef8b623efa4412ae2bfff361a0a8a8: No bootstrap required, opened a new log
I20260923 15:02:01.581764 20983 ts_tablet_manager.cc:1397] T bf6189755d9f4d72966ef3aef88aa391 P e9ef8b623efa4412ae2bfff361a0a8a8: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 15:02:01.582572 20982 tablet_bootstrap.cc:492] T bf6189755d9f4d72966ef3aef88aa391 P 430f9682c1fe49f7b9fc238400127887: No bootstrap required, opened a new log
I20260923 15:02:01.582643 20982 ts_tablet_manager.cc:1397] T bf6189755d9f4d72966ef3aef88aa391 P 430f9682c1fe49f7b9fc238400127887: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 15:02:01.583101 20981 raft_consensus.cc:348] T bf6189755d9f4d72966ef3aef88aa391 P a32e8b9f779e4b9288aafcdb2e4a1c7a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9ef8b623efa4412ae2bfff361a0a8a8" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 42411 } } peers { permanent_uuid: "a32e8b9f779e4b9288aafcdb2e4a1c7a" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 42895 } } peers { permanent_uuid: "430f9682c1fe49f7b9fc238400127887" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 35617 } }
I20260923 15:02:01.583268 20981 raft_consensus.cc:374] T bf6189755d9f4d72966ef3aef88aa391 P a32e8b9f779e4b9288aafcdb2e4a1c7a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:02:01.583302 20981 raft_consensus.cc:729] T bf6189755d9f4d72966ef3aef88aa391 P a32e8b9f779e4b9288aafcdb2e4a1c7a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a32e8b9f779e4b9288aafcdb2e4a1c7a, State: Initialized, Role: FOLLOWER
I20260923 15:02:01.583431 20981 consensus_queue.cc:260] T bf6189755d9f4d72966ef3aef88aa391 P a32e8b9f779e4b9288aafcdb2e4a1c7a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9ef8b623efa4412ae2bfff361a0a8a8" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 42411 } } peers { permanent_uuid: "a32e8b9f779e4b9288aafcdb2e4a1c7a" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 42895 } } peers { permanent_uuid: "430f9682c1fe49f7b9fc238400127887" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 35617 } }
I20260923 15:02:01.583875 20981 ts_tablet_manager.cc:1428] T bf6189755d9f4d72966ef3aef88aa391 P a32e8b9f779e4b9288aafcdb2e4a1c7a: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260923 15:02:01.584028 20815 heartbeater.cc:503] Master 127.13.75.190:35255 was elected leader, sending a full tablet report...
I20260923 15:02:01.584764 20982 raft_consensus.cc:348] T bf6189755d9f4d72966ef3aef88aa391 P 430f9682c1fe49f7b9fc238400127887 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9ef8b623efa4412ae2bfff361a0a8a8" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 42411 } } peers { permanent_uuid: "a32e8b9f779e4b9288aafcdb2e4a1c7a" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 42895 } } peers { permanent_uuid: "430f9682c1fe49f7b9fc238400127887" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 35617 } }
I20260923 15:02:01.584944 20982 raft_consensus.cc:374] T bf6189755d9f4d72966ef3aef88aa391 P 430f9682c1fe49f7b9fc238400127887 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:02:01.584990 20982 raft_consensus.cc:729] T bf6189755d9f4d72966ef3aef88aa391 P 430f9682c1fe49f7b9fc238400127887 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 430f9682c1fe49f7b9fc238400127887, State: Initialized, Role: FOLLOWER
I20260923 15:02:01.585100 20982 consensus_queue.cc:260] T bf6189755d9f4d72966ef3aef88aa391 P 430f9682c1fe49f7b9fc238400127887 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9ef8b623efa4412ae2bfff361a0a8a8" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 42411 } } peers { permanent_uuid: "a32e8b9f779e4b9288aafcdb2e4a1c7a" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 42895 } } peers { permanent_uuid: "430f9682c1fe49f7b9fc238400127887" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 35617 } }
I20260923 15:02:01.585382 20982 ts_tablet_manager.cc:1428] T bf6189755d9f4d72966ef3aef88aa391 P 430f9682c1fe49f7b9fc238400127887: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260923 15:02:01.585433 20951 heartbeater.cc:503] Master 127.13.75.190:35255 was elected leader, sending a full tablet report...
I20260923 15:02:01.585642 20983 raft_consensus.cc:348] T bf6189755d9f4d72966ef3aef88aa391 P e9ef8b623efa4412ae2bfff361a0a8a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9ef8b623efa4412ae2bfff361a0a8a8" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 42411 } } peers { permanent_uuid: "a32e8b9f779e4b9288aafcdb2e4a1c7a" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 42895 } } peers { permanent_uuid: "430f9682c1fe49f7b9fc238400127887" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 35617 } }
I20260923 15:02:01.585842 20983 raft_consensus.cc:374] T bf6189755d9f4d72966ef3aef88aa391 P e9ef8b623efa4412ae2bfff361a0a8a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:02:01.585909 20983 raft_consensus.cc:729] T bf6189755d9f4d72966ef3aef88aa391 P e9ef8b623efa4412ae2bfff361a0a8a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9ef8b623efa4412ae2bfff361a0a8a8, State: Initialized, Role: FOLLOWER
I20260923 15:02:01.586048 20983 consensus_queue.cc:260] T bf6189755d9f4d72966ef3aef88aa391 P e9ef8b623efa4412ae2bfff361a0a8a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9ef8b623efa4412ae2bfff361a0a8a8" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 42411 } } peers { permanent_uuid: "a32e8b9f779e4b9288aafcdb2e4a1c7a" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 42895 } } peers { permanent_uuid: "430f9682c1fe49f7b9fc238400127887" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 35617 } }
I20260923 15:02:01.586369 20983 ts_tablet_manager.cc:1428] T bf6189755d9f4d72966ef3aef88aa391 P e9ef8b623efa4412ae2bfff361a0a8a8: Time spent starting tablet: real 0.005s user 0.004s sys 0.000s
I20260923 15:02:01.586436 20678 heartbeater.cc:503] Master 127.13.75.190:35255 was elected leader, sending a full tablet report...
I20260923 15:02:01.617167 20987 raft_consensus.cc:482] T bf6189755d9f4d72966ef3aef88aa391 P a32e8b9f779e4b9288aafcdb2e4a1c7a [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 15:02:01.617429 20987 raft_consensus.cc:504] T bf6189755d9f4d72966ef3aef88aa391 P a32e8b9f779e4b9288aafcdb2e4a1c7a [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9ef8b623efa4412ae2bfff361a0a8a8" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 42411 } } peers { permanent_uuid: "a32e8b9f779e4b9288aafcdb2e4a1c7a" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 42895 } } peers { permanent_uuid: "430f9682c1fe49f7b9fc238400127887" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 35617 } }
I20260923 15:02:01.617874 20987 leader_election.cc:288] T bf6189755d9f4d72966ef3aef88aa391 P a32e8b9f779e4b9288aafcdb2e4a1c7a [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers e9ef8b623efa4412ae2bfff361a0a8a8 (127.13.75.129:42411), 430f9682c1fe49f7b9fc238400127887 (127.13.75.131:35617)
I20260923 15:02:01.621315 20819 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:01.618110 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.131:35617 (local address 127.13.75.130:37729)
0923 15:02:01.618187 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0923 15:02:01.618202 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0923 15:02:01.618259 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:02:01.618640 (+ 381us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:02:01.618642 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:02:01.618644 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:02:01.618730 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:01.618732 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:01.619557 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:01.619558 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:01.620440 (+ 882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:01.620442 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:01.621023 (+ 581us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:01.621024 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:01.621106 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:01.621109 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 15:02:01.621164 (+ 55us) client_negotiation.cc:240] Negotiation successful
0923 15:02:01.621196 (+ 32us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":48,"mutex_wait_us":23}
I20260923 15:02:01.621312 20978 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:01.618392 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.130:37729 (local address 127.13.75.131:35617)
0923 15:02:01.618459 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0923 15:02:01.618461 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:02:01.618469 (+ 8us) server_negotiation.cc:407] Connection header received
0923 15:02:01.618524 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:02:01.618525 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:02:01.618537 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:02:01.618602 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:02:01.618845 (+ 243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:01.619439 (+ 594us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:01.620577 (+ 1138us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:01.620943 (+ 366us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:01.621003 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:01.621010 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0923 15:02:01.621192 (+ 182us) server_negotiation.cc:299] Negotiation successful
0923 15:02:01.621241 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260923 15:02:01.621387 20980 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:01.618276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.130:34495 (local address 127.13.75.129:42411)
0923 15:02:01.618341 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0923 15:02:01.618343 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:02:01.618581 (+ 238us) server_negotiation.cc:407] Connection header received
0923 15:02:01.618674 (+ 93us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:02:01.618675 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:02:01.618687 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:02:01.618766 (+ 79us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:02:01.619044 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:01.619592 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:01.620523 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:01.620954 (+ 431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:01.621061 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:01.621069 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0923 15:02:01.621251 (+ 182us) server_negotiation.cc:299] Negotiation successful
0923 15:02:01.621300 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260923 15:02:01.621603 20821 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:01.618463 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.129:42411 (local address 127.13.75.130:34495)
0923 15:02:01.618525 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0923 15:02:01.618537 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 15:02:01.618592 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:02:01.618845 (+ 253us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:02:01.618846 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:02:01.618848 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:02:01.618935 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:01.618940 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:01.619678 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:01.619679 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:01.620430 (+ 751us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:01.620432 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:01.621077 (+ 645us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:01.621078 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:01.621166 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:01.621170 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 15:02:01.621228 (+ 58us) client_negotiation.cc:240] Negotiation successful
0923 15:02:01.621268 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":45,"mutex_wait_us":2}
I20260923 15:02:01.621788 20905 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf6189755d9f4d72966ef3aef88aa391" candidate_uuid: "a32e8b9f779e4b9288aafcdb2e4a1c7a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "430f9682c1fe49f7b9fc238400127887" is_pre_election: true
I20260923 15:02:01.622004 20905 raft_consensus.cc:2454] T bf6189755d9f4d72966ef3aef88aa391 P 430f9682c1fe49f7b9fc238400127887 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a32e8b9f779e4b9288aafcdb2e4a1c7a in term 0.
I20260923 15:02:01.621965 20632 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf6189755d9f4d72966ef3aef88aa391" candidate_uuid: "a32e8b9f779e4b9288aafcdb2e4a1c7a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9ef8b623efa4412ae2bfff361a0a8a8" is_pre_election: true
I20260923 15:02:01.622145 20632 raft_consensus.cc:2454] T bf6189755d9f4d72966ef3aef88aa391 P e9ef8b623efa4412ae2bfff361a0a8a8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate a32e8b9f779e4b9288aafcdb2e4a1c7a in term 0.
I20260923 15:02:01.622265 20704 leader_election.cc:302] T bf6189755d9f4d72966ef3aef88aa391 P a32e8b9f779e4b9288aafcdb2e4a1c7a [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 430f9682c1fe49f7b9fc238400127887, a32e8b9f779e4b9288aafcdb2e4a1c7a; no voters:
I20260923 15:02:01.622424 20987 raft_consensus.cc:2781] T bf6189755d9f4d72966ef3aef88aa391 P a32e8b9f779e4b9288aafcdb2e4a1c7a [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 15:02:01.622493 20987 raft_consensus.cc:482] T bf6189755d9f4d72966ef3aef88aa391 P a32e8b9f779e4b9288aafcdb2e4a1c7a [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 15:02:01.622509 20987 raft_consensus.cc:3037] T bf6189755d9f4d72966ef3aef88aa391 P a32e8b9f779e4b9288aafcdb2e4a1c7a [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:02:01.623265 20987 raft_consensus.cc:504] T bf6189755d9f4d72966ef3aef88aa391 P a32e8b9f779e4b9288aafcdb2e4a1c7a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9ef8b623efa4412ae2bfff361a0a8a8" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 42411 } } peers { permanent_uuid: "a32e8b9f779e4b9288aafcdb2e4a1c7a" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 42895 } } peers { permanent_uuid: "430f9682c1fe49f7b9fc238400127887" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 35617 } }
I20260923 15:02:01.623574 20632 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf6189755d9f4d72966ef3aef88aa391" candidate_uuid: "a32e8b9f779e4b9288aafcdb2e4a1c7a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e9ef8b623efa4412ae2bfff361a0a8a8"
I20260923 15:02:01.623662 20632 raft_consensus.cc:3037] T bf6189755d9f4d72966ef3aef88aa391 P e9ef8b623efa4412ae2bfff361a0a8a8 [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:02:01.623921 20987 leader_election.cc:288] T bf6189755d9f4d72966ef3aef88aa391 P a32e8b9f779e4b9288aafcdb2e4a1c7a [CANDIDATE]: Term 1 election: Requested vote from peers e9ef8b623efa4412ae2bfff361a0a8a8 (127.13.75.129:42411), 430f9682c1fe49f7b9fc238400127887 (127.13.75.131:35617)
I20260923 15:02:01.623951 20905 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bf6189755d9f4d72966ef3aef88aa391" candidate_uuid: "a32e8b9f779e4b9288aafcdb2e4a1c7a" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "430f9682c1fe49f7b9fc238400127887"
I20260923 15:02:01.624083 20905 raft_consensus.cc:3037] T bf6189755d9f4d72966ef3aef88aa391 P 430f9682c1fe49f7b9fc238400127887 [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:02:01.624595 20632 raft_consensus.cc:2454] T bf6189755d9f4d72966ef3aef88aa391 P e9ef8b623efa4412ae2bfff361a0a8a8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a32e8b9f779e4b9288aafcdb2e4a1c7a in term 1.
I20260923 15:02:01.624814 20702 leader_election.cc:302] T bf6189755d9f4d72966ef3aef88aa391 P a32e8b9f779e4b9288aafcdb2e4a1c7a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a32e8b9f779e4b9288aafcdb2e4a1c7a, e9ef8b623efa4412ae2bfff361a0a8a8; no voters:
I20260923 15:02:01.624917 20987 raft_consensus.cc:2781] T bf6189755d9f4d72966ef3aef88aa391 P a32e8b9f779e4b9288aafcdb2e4a1c7a [term 1 FOLLOWER]: Leader election won for term 1
I20260923 15:02:01.624989 20905 raft_consensus.cc:2454] T bf6189755d9f4d72966ef3aef88aa391 P 430f9682c1fe49f7b9fc238400127887 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate a32e8b9f779e4b9288aafcdb2e4a1c7a in term 1.
I20260923 15:02:01.625103 20987 raft_consensus.cc:686] T bf6189755d9f4d72966ef3aef88aa391 P a32e8b9f779e4b9288aafcdb2e4a1c7a [term 1 LEADER]: Becoming Leader. State: Replica: a32e8b9f779e4b9288aafcdb2e4a1c7a, State: Running, Role: LEADER
W20260923 15:02:01.625135 20818 tablet.cc:2369] T bf6189755d9f4d72966ef3aef88aa391 P a32e8b9f779e4b9288aafcdb2e4a1c7a: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 15:02:01.625212 20987 consensus_queue.cc:237] T bf6189755d9f4d72966ef3aef88aa391 P a32e8b9f779e4b9288aafcdb2e4a1c7a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9ef8b623efa4412ae2bfff361a0a8a8" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 42411 } } peers { permanent_uuid: "a32e8b9f779e4b9288aafcdb2e4a1c7a" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 42895 } } peers { permanent_uuid: "430f9682c1fe49f7b9fc238400127887" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 35617 } }
I20260923 15:02:01.626214 19709 catalog_manager.cc:5495] T bf6189755d9f4d72966ef3aef88aa391 P a32e8b9f779e4b9288aafcdb2e4a1c7a reported cstate change: term changed from 0 to 1, leader changed from <none> to a32e8b9f779e4b9288aafcdb2e4a1c7a (127.13.75.130). New cstate: current_term: 1 leader_uuid: "a32e8b9f779e4b9288aafcdb2e4a1c7a" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9ef8b623efa4412ae2bfff361a0a8a8" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 42411 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a32e8b9f779e4b9288aafcdb2e4a1c7a" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 42895 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "430f9682c1fe49f7b9fc238400127887" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 35617 } health_report { overall_health: UNKNOWN } } }
I20260923 15:02:01.636077 20979 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:01.633379 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60136 (local address 127.13.75.130:42895)
0923 15:02:01.633451 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0923 15:02:01.633453 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:02:01.633463 (+ 10us) server_negotiation.cc:407] Connection header received
0923 15:02:01.633507 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:02:01.633508 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:02:01.633696 (+ 188us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:02:01.633779 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 15:02:01.633924 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:01.634399 (+ 475us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:01.635328 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:01.635533 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:01.635599 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:01.635736 (+ 137us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 15:02:01.635840 (+ 104us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 15:02:01.635902 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0923 15:02:01.635947 (+ 45us) server_negotiation.cc:299] Negotiation successful
0923 15:02:01.635969 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260923 15:02:01.638106 20905 raft_consensus.cc:1261] T bf6189755d9f4d72966ef3aef88aa391 P 430f9682c1fe49f7b9fc238400127887 [term 1 FOLLOWER]: Refusing update from remote peer a32e8b9f779e4b9288aafcdb2e4a1c7a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 15:02:01.638108 20632 raft_consensus.cc:1261] T bf6189755d9f4d72966ef3aef88aa391 P e9ef8b623efa4412ae2bfff361a0a8a8 [term 1 FOLLOWER]: Refusing update from remote peer a32e8b9f779e4b9288aafcdb2e4a1c7a: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 15:02:01.638509 20987 consensus_queue.cc:1035] T bf6189755d9f4d72966ef3aef88aa391 P a32e8b9f779e4b9288aafcdb2e4a1c7a [LEADER]: Connected to new peer: Peer: permanent_uuid: "430f9682c1fe49f7b9fc238400127887" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 35617 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 15:02:01.638672 20990 consensus_queue.cc:1035] T bf6189755d9f4d72966ef3aef88aa391 P a32e8b9f779e4b9288aafcdb2e4a1c7a [LEADER]: Connected to new peer: Peer: permanent_uuid: "e9ef8b623efa4412ae2bfff361a0a8a8" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 42411 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 15:02:01.643412 20995 mvcc.cc:205] Tried to move back new op lower bound from 7332559755826315264 to 7332559755777941504. Current Snapshot: MvccSnapshot[applied={T|T < 7332559755826315264}]
I20260923 15:02:01.644203 20993 mvcc.cc:205] Tried to move back new op lower bound from 7332559755826315264 to 7332559755777941504. Current Snapshot: MvccSnapshot[applied={T|T < 7332559755826315264 or (T in {7332559755826315264})}]
I20260923 15:02:01.645375 19709 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50810:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 15:02:01.645473 19709 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:50810:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 15:02:01.647024 19709 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 176378cdc0584ce69eca703063507558: Sending DeleteTablet for 3 replicas of tablet bf6189755d9f4d72966ef3aef88aa391
I20260923 15:02:01.647446 20612 tablet_service.cc:1514] Processing DeleteTablet for tablet bf6189755d9f4d72966ef3aef88aa391 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 15:02:01 UTC) from {username='oryx', principal='oryx/127.13.75.190@KRBTEST.COM'} at 127.0.0.1:51176
I20260923 15:02:01.647488 20749 tablet_service.cc:1514] Processing DeleteTablet for tablet bf6189755d9f4d72966ef3aef88aa391 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 15:02:01 UTC) from {username='oryx', principal='oryx/127.13.75.190@KRBTEST.COM'} at 127.0.0.1:60130
I20260923 15:02:01.647697 21003 tablet_replica.cc:330] T bf6189755d9f4d72966ef3aef88aa391 P e9ef8b623efa4412ae2bfff361a0a8a8: stopping tablet replica
I20260923 15:02:01.647830 21003 raft_consensus.cc:2229] T bf6189755d9f4d72966ef3aef88aa391 P e9ef8b623efa4412ae2bfff361a0a8a8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 15:02:01.648183 20885 tablet_service.cc:1514] Processing DeleteTablet for tablet bf6189755d9f4d72966ef3aef88aa391 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 15:02:01 UTC) from {username='oryx', principal='oryx/127.13.75.190@KRBTEST.COM'} at 127.0.0.1:55026
I20260923 15:02:01.648231 21003 raft_consensus.cc:2258] T bf6189755d9f4d72966ef3aef88aa391 P e9ef8b623efa4412ae2bfff361a0a8a8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 15:02:01.648689 21004 tablet_replica.cc:330] T bf6189755d9f4d72966ef3aef88aa391 P a32e8b9f779e4b9288aafcdb2e4a1c7a: stopping tablet replica
I20260923 15:02:01.648746 13614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKC4soy/build/release/bin/kudu with pid 19677
I20260923 15:02:01.648792 21004 raft_consensus.cc:2229] T bf6189755d9f4d72966ef3aef88aa391 P a32e8b9f779e4b9288aafcdb2e4a1c7a [term 1 LEADER]: Raft consensus shutting down.
I20260923 15:02:01.648885 21004 raft_consensus.cc:2258] T bf6189755d9f4d72966ef3aef88aa391 P a32e8b9f779e4b9288aafcdb2e4a1c7a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 15:02:01.648809 21003 ts_tablet_manager.cc:1905] T bf6189755d9f4d72966ef3aef88aa391 P e9ef8b623efa4412ae2bfff361a0a8a8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 15:02:01.650357 21004 ts_tablet_manager.cc:1905] T bf6189755d9f4d72966ef3aef88aa391 P a32e8b9f779e4b9288aafcdb2e4a1c7a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 15:02:01.651367 21003 ts_tablet_manager.cc:1918] T bf6189755d9f4d72966ef3aef88aa391 P e9ef8b623efa4412ae2bfff361a0a8a8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 15:02:01.651440 21003 log.cc:1196] T bf6189755d9f4d72966ef3aef88aa391 P e9ef8b623efa4412ae2bfff361a0a8a8: Deleting WAL directory at /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/minicluster-data/ts-0/wal/wals/bf6189755d9f4d72966ef3aef88aa391
I20260923 15:02:01.651697 21003 ts_tablet_manager.cc:1939] T bf6189755d9f4d72966ef3aef88aa391 P e9ef8b623efa4412ae2bfff361a0a8a8: Deleting consensus metadata
I20260923 15:02:01.652110 21006 tablet_replica.cc:330] T bf6189755d9f4d72966ef3aef88aa391 P 430f9682c1fe49f7b9fc238400127887: stopping tablet replica
I20260923 15:02:01.652118 21004 ts_tablet_manager.cc:1918] T bf6189755d9f4d72966ef3aef88aa391 P a32e8b9f779e4b9288aafcdb2e4a1c7a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 15:02:01.652182 21004 log.cc:1196] T bf6189755d9f4d72966ef3aef88aa391 P a32e8b9f779e4b9288aafcdb2e4a1c7a: Deleting WAL directory at /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/minicluster-data/ts-1/wal/wals/bf6189755d9f4d72966ef3aef88aa391
I20260923 15:02:01.652225 21006 raft_consensus.cc:2229] T bf6189755d9f4d72966ef3aef88aa391 P 430f9682c1fe49f7b9fc238400127887 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 15:02:01.652334 21006 raft_consensus.cc:2258] T bf6189755d9f4d72966ef3aef88aa391 P 430f9682c1fe49f7b9fc238400127887 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 15:02:01.652453 21004 ts_tablet_manager.cc:1939] T bf6189755d9f4d72966ef3aef88aa391 P a32e8b9f779e4b9288aafcdb2e4a1c7a: Deleting consensus metadata
W20260923 15:02:01.655923 20705 connection.cc:535] server connection from 127.0.0.1:60130 recv error: Network error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260923 15:02:01.655977 21006 ts_tablet_manager.cc:1905] T bf6189755d9f4d72966ef3aef88aa391 P 430f9682c1fe49f7b9fc238400127887: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 15:02:01.657984 21006 ts_tablet_manager.cc:1918] T bf6189755d9f4d72966ef3aef88aa391 P 430f9682c1fe49f7b9fc238400127887: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 15:02:01.658164 21006 log.cc:1196] T bf6189755d9f4d72966ef3aef88aa391 P 430f9682c1fe49f7b9fc238400127887: Deleting WAL directory at /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/minicluster-data/ts-2/wal/wals/bf6189755d9f4d72966ef3aef88aa391
I20260923 15:02:01.658488 21006 ts_tablet_manager.cc:1939] T bf6189755d9f4d72966ef3aef88aa391 P 430f9682c1fe49f7b9fc238400127887: Deleting consensus metadata
W20260923 15:02:01.658910 20840 connection.cc:441] server connection from 127.0.0.1:55026 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:55026 (request call id 1) could send its response
I20260923 15:02:01.663619 20958 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:01.648308 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41058 (local address 127.13.75.188:33585)
0923 15:02:01.648350 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0923 15:02:01.648352 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:02:01.655295 (+ 6943us) server_negotiation.cc:407] Connection header received
0923 15:02:01.655333 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:02:01.655334 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:02:01.655349 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:02:01.655421 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:02:01.656192 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:01.656986 (+ 794us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:01.658079 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:01.658397 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:01.658440 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:01.660250 (+ 1810us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:02:01.660252 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:02:01.660252 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:02:01.660262 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:02:01.662204 (+ 1942us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:02:01.662656 (+ 452us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:02:01.662657 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:02:01.662658 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:02:01.662718 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:02:01.662948 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:02:01.662949 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:02:01.662950 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:02:01.663094 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:02:01.663201 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0923 15:02:01.663377 (+ 176us) server_negotiation.cc:299] Negotiation successful
0923 15:02:01.663489 (+ 112us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
W20260923 15:02:01.664549 20568 connection.cc:441] server connection from 127.0.0.1:51176 torn down before Call kudu.tserver.TabletServerAdminService.DeleteTablet from 127.0.0.1:51176 (request call id 1) could send its response
I20260923 15:02:01.667009 20966 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:01.659096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39198 (local address 127.13.75.189:44251)
0923 15:02:01.659143 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0923 15:02:01.659145 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:02:01.659435 (+ 290us) server_negotiation.cc:407] Connection header received
0923 15:02:01.659518 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:02:01.659519 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:02:01.659536 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:02:01.659586 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:02:01.660400 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:01.661160 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:01.662088 (+ 928us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:01.662308 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:01.662402 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:01.663875 (+ 1473us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:02:01.663876 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:02:01.663877 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:02:01.663885 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:02:01.665759 (+ 1874us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:02:01.666124 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:02:01.666125 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:02:01.666125 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:02:01.666178 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:02:01.666398 (+ 220us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:02:01.666399 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:02:01.666400 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:02:01.666525 (+ 125us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:02:01.666600 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0923 15:02:01.666794 (+ 194us) server_negotiation.cc:299] Negotiation successful
0923 15:02:01.666910 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260923 15:02:01.672096 20682 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:01.661302 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41074 (local address 127.13.75.188:33585)
0923 15:02:01.661357 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0923 15:02:01.661360 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:02:01.664635 (+ 3275us) server_negotiation.cc:407] Connection header received
0923 15:02:01.664778 (+ 143us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:02:01.664781 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:02:01.664823 (+ 42us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:02:01.664929 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:02:01.665555 (+ 626us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:01.666318 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:01.667369 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:01.667658 (+ 289us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:01.667699 (+ 41us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:01.669359 (+ 1660us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:02:01.669360 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:02:01.669360 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:02:01.669368 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:02:01.670801 (+ 1433us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:02:01.671361 (+ 560us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:02:01.671362 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:02:01.671363 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:02:01.671419 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:02:01.671643 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:02:01.671643 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:02:01.671643 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:02:01.671750 (+ 107us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:02:01.671826 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0923 15:02:01.671961 (+ 135us) server_negotiation.cc:299] Negotiation successful
0923 15:02:01.672028 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260923 15:02:02.643720 21014 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:02.643312 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:35255 (local address 127.13.75.129:58223)
0923 15:02:02.643562 (+ 250us) negotiation.cc:107] Waiting for socket to connect
0923 15:02:02.643659 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.190:35255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":225,"thread_start_us":162,"threads_started":1}
W20260923 15:02:02.644045 20678 heartbeater.cc:644] Failed to heartbeat to 127.13.75.190:35255 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.13.75.190:35255: connect: Connection refused (error 111)
I20260923 15:02:02.644593 21015 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:02.644361 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:35255 (local address 127.13.75.130:40597)
0923 15:02:02.644474 (+ 113us) negotiation.cc:107] Waiting for socket to connect
0923 15:02:02.644546 (+ 72us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.190:35255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":94,"thread_start_us":68,"threads_started":1}
I20260923 15:02:02.644665 21016 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:02.644401 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:35255 (local address 127.13.75.131:46659)
0923 15:02:02.644528 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0923 15:02:02.644597 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.190:35255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":62,"threads_started":1}
W20260923 15:02:02.644832 20951 heartbeater.cc:644] Failed to heartbeat to 127.13.75.190:35255 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.13.75.190:35255: connect: Connection refused (error 111)
W20260923 15:02:02.644887 20815 heartbeater.cc:644] Failed to heartbeat to 127.13.75.190:35255 (0 consecutive failures): Network error: Failed to send heartbeat to master: Client connection negotiation failed: client connection to 127.13.75.190:35255: connect: Connection refused (error 111)
I20260923 15:02:03.147739 21017 raft_consensus.cc:482] T 00000000000000000000000000000000 P b8c37de502e542aab23126e30c0e3176 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 176378cdc0584ce69eca703063507558)
I20260923 15:02:03.147821 21018 raft_consensus.cc:482] T 00000000000000000000000000000000 P 28aa43d448d94ccdb6d8776e93de4fc9 [term 1 FOLLOWER]: Starting pre-election (detected failure of leader 176378cdc0584ce69eca703063507558)
I20260923 15:02:03.147847 21017 raft_consensus.cc:504] T 00000000000000000000000000000000 P b8c37de502e542aab23126e30c0e3176 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "176378cdc0584ce69eca703063507558" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 35255 } } peers { permanent_uuid: "b8c37de502e542aab23126e30c0e3176" member_type: VOTER last_known_addr { host: "127.13.75.189" port: 44251 } attrs { promote: false } } peers { permanent_uuid: "28aa43d448d94ccdb6d8776e93de4fc9" member_type: VOTER last_known_addr { host: "127.13.75.188" port: 33585 } attrs { promote: false } }
I20260923 15:02:03.147933 21018 raft_consensus.cc:504] T 00000000000000000000000000000000 P 28aa43d448d94ccdb6d8776e93de4fc9 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "176378cdc0584ce69eca703063507558" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 35255 } } peers { permanent_uuid: "b8c37de502e542aab23126e30c0e3176" member_type: VOTER last_known_addr { host: "127.13.75.189" port: 44251 } attrs { promote: false } } peers { permanent_uuid: "28aa43d448d94ccdb6d8776e93de4fc9" member_type: VOTER last_known_addr { host: "127.13.75.188" port: 33585 } attrs { promote: false } }
I20260923 15:02:03.148077 21017 leader_election.cc:288] T 00000000000000000000000000000000 P b8c37de502e542aab23126e30c0e3176 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 176378cdc0584ce69eca703063507558 (127.13.75.190:35255), 28aa43d448d94ccdb6d8776e93de4fc9 (127.13.75.188:33585)
I20260923 15:02:03.148459 21019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:03.148223 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:35255 (local address 127.0.0.1:49068)
0923 15:02:03.148323 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0923 15:02:03.148412 (+ 89us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.190:35255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81,"thread_start_us":42,"threads_started":1}
W20260923 15:02:03.148648 20295 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.13.75.190:35255: connect: Connection refused (error 111)
I20260923 15:02:03.148857 21022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:03.148642 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:35255 (local address 127.0.0.1:49070)
0923 15:02:03.148760 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0923 15:02:03.148818 (+ 58us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.190:35255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":46,"threads_started":1}
W20260923 15:02:03.148983 20484 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.13.75.190:35255: connect: Connection refused (error 111)
I20260923 15:02:03.149173 21018 leader_election.cc:288] T 00000000000000000000000000000000 P 28aa43d448d94ccdb6d8776e93de4fc9 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 176378cdc0584ce69eca703063507558 (127.13.75.190:35255), b8c37de502e542aab23126e30c0e3176 (127.13.75.189:44251)
I20260923 15:02:03.149220 21019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:03.148970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:35255 (local address 127.0.0.1:49072)
0923 15:02:03.149156 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0923 15:02:03.149188 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.190:35255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":167}
W20260923 15:02:03.149389 20295 leader_election.cc:334] T 00000000000000000000000000000000 P b8c37de502e542aab23126e30c0e3176 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 176378cdc0584ce69eca703063507558 (127.13.75.190:35255): Network error: Client connection negotiation failed: client connection to 127.13.75.190:35255: connect: Connection refused (error 111)
I20260923 15:02:03.149559 21024 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:03.149438 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:35255 (local address 127.0.0.1:49080)
0923 15:02:03.149481 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0923 15:02:03.149529 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.190:35255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260923 15:02:03.149662 20484 leader_election.cc:334] T 00000000000000000000000000000000 P 28aa43d448d94ccdb6d8776e93de4fc9 [CANDIDATE]: Term 2 pre-election: RPC error from VoteRequest() call to peer 176378cdc0584ce69eca703063507558 (127.13.75.190:35255): Network error: Client connection negotiation failed: client connection to 127.13.75.190:35255: connect: Connection refused (error 111)
I20260923 15:02:03.152354 21020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:03.148347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.188:33585 (local address 127.0.0.1:36600)
0923 15:02:03.148445 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0923 15:02:03.148457 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 15:02:03.148775 (+ 318us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:02:03.149025 (+ 250us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:02:03.149026 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:02:03.149029 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:02:03.149097 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:03.149099 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:03.150073 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:03.150074 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:03.151372 (+ 1298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:03.151376 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:03.152111 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:03.152112 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:03.152160 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:03.152165 (+ 5us) client_negotiation.cc:769] Sending connection context
0923 15:02:03.152214 (+ 49us) client_negotiation.cc:240] Negotiation successful
0923 15:02:03.152251 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":81,"thread_start_us":61,"threads_started":1}
I20260923 15:02:03.152355 21021 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:03.148530 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36600 (local address 127.13.75.188:33585)
0923 15:02:03.148875 (+ 345us) server_negotiation.cc:206] Beginning negotiation
0923 15:02:03.148877 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:02:03.148885 (+ 8us) server_negotiation.cc:407] Connection header received
0923 15:02:03.148911 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:02:03.148912 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:02:03.148924 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:02:03.149169 (+ 245us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:02:03.149207 (+ 38us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:03.149986 (+ 779us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:03.151464 (+ 1478us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:03.151988 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:03.152093 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:03.152106 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0923 15:02:03.152231 (+ 125us) server_negotiation.cc:299] Negotiation successful
0923 15:02:03.152284 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":326,"thread_start_us":45,"threads_started":1}
I20260923 15:02:03.152539 21022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:03.148956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.189:44251 (local address 127.0.0.1:36056)
0923 15:02:03.149170 (+ 214us) negotiation.cc:107] Waiting for socket to connect
0923 15:02:03.149180 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 15:02:03.149238 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:02:03.149636 (+ 398us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:02:03.149636 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:02:03.149639 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:02:03.149675 (+ 36us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:03.149676 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:03.150415 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:03.150416 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:03.151591 (+ 1175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:03.151594 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:03.152311 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:03.152312 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:03.152372 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:03.152377 (+ 5us) client_negotiation.cc:769] Sending connection context
0923 15:02:03.152431 (+ 54us) client_negotiation.cc:240] Negotiation successful
0923 15:02:03.152477 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":205,"thread_start_us":45,"threads_started":1}
I20260923 15:02:03.152710 21026 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:03.149076 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:36056 (local address 127.13.75.189:44251)
0923 15:02:03.149305 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0923 15:02:03.149307 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:02:03.149313 (+ 6us) server_negotiation.cc:407] Connection header received
0923 15:02:03.149333 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:02:03.149333 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:02:03.149343 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:02:03.149740 (+ 397us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:02:03.149774 (+ 34us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:03.150346 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:03.151667 (+ 1321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:03.152230 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:03.152297 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:03.152308 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0923 15:02:03.152617 (+ 309us) server_negotiation.cc:299] Negotiation successful
0923 15:02:03.152663 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":207,"thread_start_us":75,"threads_started":1}
I20260923 15:02:03.152654 20509 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b8c37de502e542aab23126e30c0e3176" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "28aa43d448d94ccdb6d8776e93de4fc9" is_pre_election: true
I20260923 15:02:03.152778 20509 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 28aa43d448d94ccdb6d8776e93de4fc9 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8c37de502e542aab23126e30c0e3176 in term 1.
I20260923 15:02:03.152917 20295 leader_election.cc:302] T 00000000000000000000000000000000 P b8c37de502e542aab23126e30c0e3176 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28aa43d448d94ccdb6d8776e93de4fc9, b8c37de502e542aab23126e30c0e3176; no voters: 176378cdc0584ce69eca703063507558
I20260923 15:02:03.152879 20320 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "28aa43d448d94ccdb6d8776e93de4fc9" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "b8c37de502e542aab23126e30c0e3176" is_pre_election: true
I20260923 15:02:03.152987 20320 raft_consensus.cc:2454] T 00000000000000000000000000000000 P b8c37de502e542aab23126e30c0e3176 [term 1 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 28aa43d448d94ccdb6d8776e93de4fc9 in term 1.
I20260923 15:02:03.153069 21017 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b8c37de502e542aab23126e30c0e3176 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260923 15:02:03.153105 21017 raft_consensus.cc:482] T 00000000000000000000000000000000 P b8c37de502e542aab23126e30c0e3176 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 176378cdc0584ce69eca703063507558)
I20260923 15:02:03.153131 21017 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b8c37de502e542aab23126e30c0e3176 [term 1 FOLLOWER]: Advancing to term 2
I20260923 15:02:03.153132 20485 leader_election.cc:302] T 00000000000000000000000000000000 P 28aa43d448d94ccdb6d8776e93de4fc9 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28aa43d448d94ccdb6d8776e93de4fc9, b8c37de502e542aab23126e30c0e3176; no voters: 176378cdc0584ce69eca703063507558
I20260923 15:02:03.153240 21018 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 28aa43d448d94ccdb6d8776e93de4fc9 [term 1 FOLLOWER]: Leader pre-election won for term 2
I20260923 15:02:03.153270 21018 raft_consensus.cc:482] T 00000000000000000000000000000000 P 28aa43d448d94ccdb6d8776e93de4fc9 [term 1 FOLLOWER]: Starting leader election (detected failure of leader 176378cdc0584ce69eca703063507558)
I20260923 15:02:03.153324 21018 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 28aa43d448d94ccdb6d8776e93de4fc9 [term 1 FOLLOWER]: Advancing to term 2
I20260923 15:02:03.153904 21017 raft_consensus.cc:504] T 00000000000000000000000000000000 P b8c37de502e542aab23126e30c0e3176 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "176378cdc0584ce69eca703063507558" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 35255 } } peers { permanent_uuid: "b8c37de502e542aab23126e30c0e3176" member_type: VOTER last_known_addr { host: "127.13.75.189" port: 44251 } attrs { promote: false } } peers { permanent_uuid: "28aa43d448d94ccdb6d8776e93de4fc9" member_type: VOTER last_known_addr { host: "127.13.75.188" port: 33585 } attrs { promote: false } }
I20260923 15:02:03.154049 21017 leader_election.cc:288] T 00000000000000000000000000000000 P b8c37de502e542aab23126e30c0e3176 [CANDIDATE]: Term 2 election: Requested vote from peers 176378cdc0584ce69eca703063507558 (127.13.75.190:35255), 28aa43d448d94ccdb6d8776e93de4fc9 (127.13.75.188:33585)
I20260923 15:02:03.154093 21018 raft_consensus.cc:504] T 00000000000000000000000000000000 P 28aa43d448d94ccdb6d8776e93de4fc9 [term 2 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "176378cdc0584ce69eca703063507558" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 35255 } } peers { permanent_uuid: "b8c37de502e542aab23126e30c0e3176" member_type: VOTER last_known_addr { host: "127.13.75.189" port: 44251 } attrs { promote: false } } peers { permanent_uuid: "28aa43d448d94ccdb6d8776e93de4fc9" member_type: VOTER last_known_addr { host: "127.13.75.188" port: 33585 } attrs { promote: false } }
I20260923 15:02:03.154229 21018 leader_election.cc:288] T 00000000000000000000000000000000 P 28aa43d448d94ccdb6d8776e93de4fc9 [CANDIDATE]: Term 2 election: Requested vote from peers 176378cdc0584ce69eca703063507558 (127.13.75.190:35255), b8c37de502e542aab23126e30c0e3176 (127.13.75.189:44251)
I20260923 15:02:03.154299 21020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:03.154177 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:35255 (local address 127.0.0.1:49096)
0923 15:02:03.154209 (+ 32us) negotiation.cc:107] Waiting for socket to connect
0923 15:02:03.154274 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.190:35255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":15}
I20260923 15:02:03.154409 20320 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "28aa43d448d94ccdb6d8776e93de4fc9" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "b8c37de502e542aab23126e30c0e3176"
I20260923 15:02:03.154414 20509 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b8c37de502e542aab23126e30c0e3176" candidate_term: 2 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "28aa43d448d94ccdb6d8776e93de4fc9"
I20260923 15:02:03.154472 21022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:03.154310 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:35255 (local address 127.0.0.1:49102)
0923 15:02:03.154382 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0923 15:02:03.154412 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.190:35255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":58}
I20260923 15:02:03.154489 20320 raft_consensus.cc:2379] T 00000000000000000000000000000000 P b8c37de502e542aab23126e30c0e3176 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate 28aa43d448d94ccdb6d8776e93de4fc9 in current term 2: Already voted for candidate b8c37de502e542aab23126e30c0e3176 in this term.
I20260923 15:02:03.154510 20509 raft_consensus.cc:2379] T 00000000000000000000000000000000 P 28aa43d448d94ccdb6d8776e93de4fc9 [term 2 FOLLOWER]: Leader election vote request: Denying vote to candidate b8c37de502e542aab23126e30c0e3176 in current term 2: Already voted for candidate 28aa43d448d94ccdb6d8776e93de4fc9 in this term.
I20260923 15:02:03.154635 21020 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:03.154527 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:35255 (local address 127.0.0.1:49118)
0923 15:02:03.154594 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0923 15:02:03.154617 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.190:35255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260923 15:02:03.154701 20295 leader_election.cc:334] T 00000000000000000000000000000000 P b8c37de502e542aab23126e30c0e3176 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 176378cdc0584ce69eca703063507558 (127.13.75.190:35255): Network error: Client connection negotiation failed: client connection to 127.13.75.190:35255: connect: Connection refused (error 111)
I20260923 15:02:03.154788 20295 leader_election.cc:302] T 00000000000000000000000000000000 P b8c37de502e542aab23126e30c0e3176 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b8c37de502e542aab23126e30c0e3176; no voters: 176378cdc0584ce69eca703063507558, 28aa43d448d94ccdb6d8776e93de4fc9
I20260923 15:02:03.154903 21017 raft_consensus.cc:2726] T 00000000000000000000000000000000 P b8c37de502e542aab23126e30c0e3176 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260923 15:02:03.155005 21022 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:03.154810 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:35255 (local address 127.0.0.1:49132)
0923 15:02:03.154925 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0923 15:02:03.154982 (+ 57us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.190:35255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":98}
W20260923 15:02:03.155090 20484 leader_election.cc:334] T 00000000000000000000000000000000 P 28aa43d448d94ccdb6d8776e93de4fc9 [CANDIDATE]: Term 2 election: RPC error from VoteRequest() call to peer 176378cdc0584ce69eca703063507558 (127.13.75.190:35255): Network error: Client connection negotiation failed: client connection to 127.13.75.190:35255: connect: Connection refused (error 111)
I20260923 15:02:03.155129 20484 leader_election.cc:302] T 00000000000000000000000000000000 P 28aa43d448d94ccdb6d8776e93de4fc9 [CANDIDATE]: Term 2 election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 28aa43d448d94ccdb6d8776e93de4fc9; no voters: 176378cdc0584ce69eca703063507558, b8c37de502e542aab23126e30c0e3176
I20260923 15:02:03.155203 21018 raft_consensus.cc:2726] T 00000000000000000000000000000000 P 28aa43d448d94ccdb6d8776e93de4fc9 [term 2 FOLLOWER]: Leader election lost for term 2. Reason: could not achieve majority
I20260923 15:02:03.644735 21027 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:03.644429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:35255 (local address 127.13.75.129:48691)
0923 15:02:03.644616 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0923 15:02:03.644677 (+ 61us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.190:35255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":167,"thread_start_us":109,"threads_started":1}
I20260923 15:02:03.645326 21028 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:03.645097 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:35255 (local address 127.13.75.131:57015)
0923 15:02:03.645243 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0923 15:02:03.645277 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.190:35255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":60,"threads_started":1}
I20260923 15:02:03.645370 21029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:03.645165 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:35255 (local address 127.13.75.130:46449)
0923 15:02:03.645281 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0923 15:02:03.645317 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.190:35255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":58,"threads_started":1}
I20260923 15:02:04.645687 21030 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:04.645443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:35255 (local address 127.13.75.129:38017)
0923 15:02:04.645566 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0923 15:02:04.645609 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.190:35255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":70,"threads_started":1}
I20260923 15:02:04.646063 21031 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:04.645855 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:35255 (local address 127.13.75.131:53681)
0923 15:02:04.645982 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0923 15:02:04.646020 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.190:35255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":50,"threads_started":1}
I20260923 15:02:04.646081 21032 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:04.645914 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:35255 (local address 127.13.75.130:60429)
0923 15:02:04.645996 (+ 82us) negotiation.cc:107] Waiting for socket to connect
0923 15:02:04.646034 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.190:35255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":64,"thread_start_us":42,"threads_started":1}
I20260923 15:02:05.177695 21033 raft_consensus.cc:482] T 00000000000000000000000000000000 P b8c37de502e542aab23126e30c0e3176 [term 2 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 15:02:05.177807 21033 raft_consensus.cc:504] T 00000000000000000000000000000000 P b8c37de502e542aab23126e30c0e3176 [term 2 FOLLOWER]: Starting pre-election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "176378cdc0584ce69eca703063507558" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 35255 } } peers { permanent_uuid: "b8c37de502e542aab23126e30c0e3176" member_type: VOTER last_known_addr { host: "127.13.75.189" port: 44251 } attrs { promote: false } } peers { permanent_uuid: "28aa43d448d94ccdb6d8776e93de4fc9" member_type: VOTER last_known_addr { host: "127.13.75.188" port: 33585 } attrs { promote: false } }
I20260923 15:02:05.178033 21033 leader_election.cc:288] T 00000000000000000000000000000000 P b8c37de502e542aab23126e30c0e3176 [CANDIDATE]: Term 3 pre-election: Requested pre-vote from peers 176378cdc0584ce69eca703063507558 (127.13.75.190:35255), 28aa43d448d94ccdb6d8776e93de4fc9 (127.13.75.188:33585)
I20260923 15:02:05.178416 21034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:05.178211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:35255 (local address 127.0.0.1:49414)
0923 15:02:05.178327 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0923 15:02:05.178371 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.190:35255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":87,"thread_start_us":64,"threads_started":1}
I20260923 15:02:05.178525 20509 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b8c37de502e542aab23126e30c0e3176" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "28aa43d448d94ccdb6d8776e93de4fc9" is_pre_election: true
I20260923 15:02:05.178629 20509 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 28aa43d448d94ccdb6d8776e93de4fc9 [term 2 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b8c37de502e542aab23126e30c0e3176 in term 2.
I20260923 15:02:05.179068 20295 leader_election.cc:302] T 00000000000000000000000000000000 P b8c37de502e542aab23126e30c0e3176 [CANDIDATE]: Term 3 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 28aa43d448d94ccdb6d8776e93de4fc9, b8c37de502e542aab23126e30c0e3176; no voters:
I20260923 15:02:05.179118 21034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:05.179008 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:35255 (local address 127.0.0.1:49424)
0923 15:02:05.179064 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0923 15:02:05.179093 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.190:35255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
W20260923 15:02:05.179171 20295 leader_election.cc:334] T 00000000000000000000000000000000 P b8c37de502e542aab23126e30c0e3176 [CANDIDATE]: Term 3 pre-election: RPC error from VoteRequest() call to peer 176378cdc0584ce69eca703063507558 (127.13.75.190:35255): Network error: Client connection negotiation failed: client connection to 127.13.75.190:35255: connect: Connection refused (error 111)
I20260923 15:02:05.179198 21033 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b8c37de502e542aab23126e30c0e3176 [term 2 FOLLOWER]: Leader pre-election won for term 3
I20260923 15:02:05.179235 21033 raft_consensus.cc:482] T 00000000000000000000000000000000 P b8c37de502e542aab23126e30c0e3176 [term 2 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 15:02:05.179273 21033 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b8c37de502e542aab23126e30c0e3176 [term 2 FOLLOWER]: Advancing to term 3
I20260923 15:02:05.180027 21033 raft_consensus.cc:504] T 00000000000000000000000000000000 P b8c37de502e542aab23126e30c0e3176 [term 3 FOLLOWER]: Starting leader election with config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "176378cdc0584ce69eca703063507558" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 35255 } } peers { permanent_uuid: "b8c37de502e542aab23126e30c0e3176" member_type: VOTER last_known_addr { host: "127.13.75.189" port: 44251 } attrs { promote: false } } peers { permanent_uuid: "28aa43d448d94ccdb6d8776e93de4fc9" member_type: VOTER last_known_addr { host: "127.13.75.188" port: 33585 } attrs { promote: false } }
I20260923 15:02:05.180205 21033 leader_election.cc:288] T 00000000000000000000000000000000 P b8c37de502e542aab23126e30c0e3176 [CANDIDATE]: Term 3 election: Requested vote from peers 176378cdc0584ce69eca703063507558 (127.13.75.190:35255), 28aa43d448d94ccdb6d8776e93de4fc9 (127.13.75.188:33585)
I20260923 15:02:05.180418 21034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:05.180337 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:35255 (local address 127.0.0.1:49428)
0923 15:02:05.180369 (+ 32us) negotiation.cc:107] Waiting for socket to connect
0923 15:02:05.180395 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.190:35255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":18}
I20260923 15:02:05.180564 20509 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "b8c37de502e542aab23126e30c0e3176" candidate_term: 3 candidate_status { last_received { term: 1 index: 12 } } ignore_live_leader: false dest_uuid: "28aa43d448d94ccdb6d8776e93de4fc9"
I20260923 15:02:05.180652 20509 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 28aa43d448d94ccdb6d8776e93de4fc9 [term 2 FOLLOWER]: Advancing to term 3
I20260923 15:02:05.180766 21034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:05.180678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:35255 (local address 127.0.0.1:49432)
0923 15:02:05.180716 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0923 15:02:05.180739 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.190:35255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":24}
W20260923 15:02:05.180855 20295 leader_election.cc:334] T 00000000000000000000000000000000 P b8c37de502e542aab23126e30c0e3176 [CANDIDATE]: Term 3 election: RPC error from VoteRequest() call to peer 176378cdc0584ce69eca703063507558 (127.13.75.190:35255): Network error: Client connection negotiation failed: client connection to 127.13.75.190:35255: connect: Connection refused (error 111)
I20260923 15:02:05.181499 20509 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 28aa43d448d94ccdb6d8776e93de4fc9 [term 3 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b8c37de502e542aab23126e30c0e3176 in term 3.
I20260923 15:02:05.181710 20295 leader_election.cc:302] T 00000000000000000000000000000000 P b8c37de502e542aab23126e30c0e3176 [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 3 responses out of 3 voters: 2 yes votes; 1 no votes. yes voters: 28aa43d448d94ccdb6d8776e93de4fc9, b8c37de502e542aab23126e30c0e3176; no voters: 176378cdc0584ce69eca703063507558
I20260923 15:02:05.181823 21033 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b8c37de502e542aab23126e30c0e3176 [term 3 FOLLOWER]: Leader election won for term 3
I20260923 15:02:05.181941 21033 raft_consensus.cc:686] T 00000000000000000000000000000000 P b8c37de502e542aab23126e30c0e3176 [term 3 LEADER]: Becoming Leader. State: Replica: b8c37de502e542aab23126e30c0e3176, State: Running, Role: LEADER
I20260923 15:02:05.182025 21033 consensus_queue.cc:237] T 00000000000000000000000000000000 P b8c37de502e542aab23126e30c0e3176 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 12, Committed index: 12, Last appended: 1.12, Last appended by leader: 12, Current term: 3, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "176378cdc0584ce69eca703063507558" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 35255 } } peers { permanent_uuid: "b8c37de502e542aab23126e30c0e3176" member_type: VOTER last_known_addr { host: "127.13.75.189" port: 44251 } attrs { promote: false } } peers { permanent_uuid: "28aa43d448d94ccdb6d8776e93de4fc9" member_type: VOTER last_known_addr { host: "127.13.75.188" port: 33585 } attrs { promote: false } }
I20260923 15:02:05.182402 21036 sys_catalog.cc:455] T 00000000000000000000000000000000 P b8c37de502e542aab23126e30c0e3176 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b8c37de502e542aab23126e30c0e3176. Latest consensus state: current_term: 3 leader_uuid: "b8c37de502e542aab23126e30c0e3176" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "176378cdc0584ce69eca703063507558" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 35255 } } peers { permanent_uuid: "b8c37de502e542aab23126e30c0e3176" member_type: VOTER last_known_addr { host: "127.13.75.189" port: 44251 } attrs { promote: false } } peers { permanent_uuid: "28aa43d448d94ccdb6d8776e93de4fc9" member_type: VOTER last_known_addr { host: "127.13.75.188" port: 33585 } attrs { promote: false } } }
I20260923 15:02:05.182511 21036 sys_catalog.cc:458] T 00000000000000000000000000000000 P b8c37de502e542aab23126e30c0e3176 [sys.catalog]: This master's current role is: LEADER
I20260923 15:02:05.182682 21038 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 15:02:05.183223 21038 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 15:02:05.183285 21038 catalog_manager.cc:1259] Loaded cluster ID: 4b5be46274e44cd197c2c043e352da13
I20260923 15:02:05.183301 21038 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 15:02:05.183523 21038 catalog_manager.cc:1504] Loading token signing keys...
I20260923 15:02:05.183597 21038 catalog_manager.cc:5879] T 00000000000000000000000000000000 P b8c37de502e542aab23126e30c0e3176: Loaded TSK: 0
I20260923 15:02:05.183744 21038 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 15:02:05.261633 20310 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39198:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 15:02:05.263438 20310 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 15:02:05.264524 21034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:05.264352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:35255 (local address 127.0.0.1:49450)
0923 15:02:05.264457 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0923 15:02:05.264496 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.190:35255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260923 15:02:05.264660 20509 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 28aa43d448d94ccdb6d8776e93de4fc9 [term 3 FOLLOWER]: Refusing update from remote peer b8c37de502e542aab23126e30c0e3176: Log matching property violated. Preceding OpId in replica: term: 1 index: 12. Preceding OpId from leader: term: 3 index: 14. (index mismatch)
I20260923 15:02:05.264909 21033 consensus_queue.cc:1035] T 00000000000000000000000000000000 P b8c37de502e542aab23126e30c0e3176 [LEADER]: Connected to new peer: Peer: permanent_uuid: "28aa43d448d94ccdb6d8776e93de4fc9" member_type: VOTER last_known_addr { host: "127.13.75.188" port: 33585 } attrs { promote: false }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 13, Last known committed idx: 12, Time since last communication: 0.000s
I20260923 15:02:05.264967 21034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:05.264814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:35255 (local address 127.0.0.1:49464)
0923 15:02:05.264869 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0923 15:02:05.264942 (+ 73us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.190:35255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
W20260923 15:02:05.265084 20295 consensus_peers.cc:487] T 00000000000000000000000000000000 P b8c37de502e542aab23126e30c0e3176 -> Peer 176378cdc0584ce69eca703063507558 (127.13.75.190:35255): Couldn't send request to peer 176378cdc0584ce69eca703063507558. Status: Network error: Client connection negotiation failed: client connection to 127.13.75.190:35255: connect: Connection refused (error 111). This is attempt 1: this message will repeat every 5th retry.
I20260923 15:02:05.265744 21040 sys_catalog.cc:455] T 00000000000000000000000000000000 P 28aa43d448d94ccdb6d8776e93de4fc9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b8c37de502e542aab23126e30c0e3176. Latest consensus state: current_term: 3 leader_uuid: "b8c37de502e542aab23126e30c0e3176" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "176378cdc0584ce69eca703063507558" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 35255 } } peers { permanent_uuid: "b8c37de502e542aab23126e30c0e3176" member_type: VOTER last_known_addr { host: "127.13.75.189" port: 44251 } attrs { promote: false } } peers { permanent_uuid: "28aa43d448d94ccdb6d8776e93de4fc9" member_type: VOTER last_known_addr { host: "127.13.75.188" port: 33585 } attrs { promote: false } } }
I20260923 15:02:05.265864 21040 sys_catalog.cc:458] T 00000000000000000000000000000000 P 28aa43d448d94ccdb6d8776e93de4fc9 [sys.catalog]: This master's current role is: FOLLOWER
I20260923 15:02:05.266443 21036 sys_catalog.cc:455] T 00000000000000000000000000000000 P b8c37de502e542aab23126e30c0e3176 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 3 leader_uuid: "b8c37de502e542aab23126e30c0e3176" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "176378cdc0584ce69eca703063507558" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 35255 } } peers { permanent_uuid: "b8c37de502e542aab23126e30c0e3176" member_type: VOTER last_known_addr { host: "127.13.75.189" port: 44251 } attrs { promote: false } } peers { permanent_uuid: "28aa43d448d94ccdb6d8776e93de4fc9" member_type: VOTER last_known_addr { host: "127.13.75.188" port: 33585 } attrs { promote: false } } }
I20260923 15:02:05.266597 21036 sys_catalog.cc:458] T 00000000000000000000000000000000 P b8c37de502e542aab23126e30c0e3176 [sys.catalog]: This master's current role is: LEADER
I20260923 15:02:05.266852 21040 sys_catalog.cc:455] T 00000000000000000000000000000000 P 28aa43d448d94ccdb6d8776e93de4fc9 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 3 leader_uuid: "b8c37de502e542aab23126e30c0e3176" committed_config { opid_index: 9 OBSOLETE_local: true peers { permanent_uuid: "176378cdc0584ce69eca703063507558" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 35255 } } peers { permanent_uuid: "b8c37de502e542aab23126e30c0e3176" member_type: VOTER last_known_addr { host: "127.13.75.189" port: 44251 } attrs { promote: false } } peers { permanent_uuid: "28aa43d448d94ccdb6d8776e93de4fc9" member_type: VOTER last_known_addr { host: "127.13.75.188" port: 33585 } attrs { promote: false } } }
I20260923 15:02:05.266992 21040 sys_catalog.cc:458] T 00000000000000000000000000000000 P 28aa43d448d94ccdb6d8776e93de4fc9 [sys.catalog]: This master's current role is: FOLLOWER
I20260923 15:02:05.272599 21047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:05.268996 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50008 (local address 127.13.75.129:42411)
0923 15:02:05.269400 (+ 404us) server_negotiation.cc:206] Beginning negotiation
0923 15:02:05.269403 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:02:05.269497 (+ 94us) server_negotiation.cc:407] Connection header received
0923 15:02:05.269531 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:02:05.269532 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:02:05.269543 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:02:05.269617 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:02:05.269730 (+ 113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:05.270297 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:05.271718 (+ 1421us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:05.272234 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:05.272308 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:05.272321 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0923 15:02:05.272463 (+ 142us) server_negotiation.cc:299] Negotiation successful
0923 15:02:05.272508 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":366,"thread_start_us":63,"threads_started":1}
I20260923 15:02:05.272607 21049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:05.268774 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.129:42411 (local address 127.0.0.1:50008)
0923 15:02:05.269422 (+ 648us) negotiation.cc:107] Waiting for socket to connect
0923 15:02:05.269430 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0923 15:02:05.269482 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:02:05.269629 (+ 147us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:02:05.269630 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:02:05.269631 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:02:05.269678 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:05.269679 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:05.270391 (+ 712us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:05.270392 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:05.271623 (+ 1231us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:05.271625 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:05.272321 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:05.272322 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:05.272381 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:05.272387 (+ 6us) client_negotiation.cc:769] Sending connection context
0923 15:02:05.272447 (+ 60us) client_negotiation.cc:240] Negotiation successful
0923 15:02:05.272506 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":631,"mutex_wait_us":199,"thread_start_us":479,"threads_started":1}
I20260923 15:02:05.272753 21034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:05.268676 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.130:42895 (local address 127.0.0.1:37420)
0923 15:02:05.268761 (+ 85us) negotiation.cc:107] Waiting for socket to connect
0923 15:02:05.268774 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0923 15:02:05.268835 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:02:05.269035 (+ 200us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:02:05.269036 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:02:05.269038 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:02:05.269102 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:05.269104 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:05.270183 (+ 1079us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:05.270184 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:05.271681 (+ 1497us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:05.271684 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:05.272554 (+ 870us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:05.272554 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:05.272608 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:05.272612 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 15:02:05.272642 (+ 30us) client_negotiation.cc:240] Negotiation successful
0923 15:02:05.272685 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":63,"mutex_wait_us":32}
I20260923 15:02:05.272940 20612 tablet_service.cc:1467] Processing CreateTablet for tablet a64d19ec71c4439ea3a00f04081ec3a3 (DEFAULT_TABLE table=test-table [id=206a01ebb8cc49fb92f7549443b8b158]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 15:02:05.272972 21048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:05.268759 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.131:35617 (local address 127.0.0.1:39008)
0923 15:02:05.269290 (+ 531us) negotiation.cc:107] Waiting for socket to connect
0923 15:02:05.269301 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 15:02:05.269347 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:02:05.269651 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:02:05.269652 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:02:05.269654 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:02:05.269714 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:05.269716 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:05.270646 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:05.270647 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:05.271922 (+ 1275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:05.271925 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:05.272715 (+ 790us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:05.272716 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:05.272787 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:05.272792 (+ 5us) client_negotiation.cc:769] Sending connection context
0923 15:02:05.272838 (+ 46us) client_negotiation.cc:240] Negotiation successful
0923 15:02:05.272886 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":508,"mutex_wait_us":1,"thread_start_us":557,"threads_started":1}
I20260923 15:02:05.273082 20612 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a64d19ec71c4439ea3a00f04081ec3a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:02:05.273193 21046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:05.268998 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39008 (local address 127.13.75.131:35617)
0923 15:02:05.269471 (+ 473us) server_negotiation.cc:206] Beginning negotiation
0923 15:02:05.269473 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:02:05.269482 (+ 9us) server_negotiation.cc:407] Connection header received
0923 15:02:05.269511 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:02:05.269512 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:02:05.269525 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:02:05.269600 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:02:05.269795 (+ 195us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:05.270501 (+ 706us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:05.272009 (+ 1508us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:05.272583 (+ 574us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:05.272666 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:05.272677 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0923 15:02:05.273062 (+ 385us) server_negotiation.cc:299] Negotiation successful
0923 15:02:05.273117 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":448,"thread_start_us":55,"threads_started":1}
I20260923 15:02:05.273415 20885 tablet_service.cc:1467] Processing CreateTablet for tablet a64d19ec71c4439ea3a00f04081ec3a3 (DEFAULT_TABLE table=test-table [id=206a01ebb8cc49fb92f7549443b8b158]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 15:02:05.273533 20885 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a64d19ec71c4439ea3a00f04081ec3a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:02:05.273660 21045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:05.268752 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37420 (local address 127.13.75.130:42895)
0923 15:02:05.268878 (+ 126us) server_negotiation.cc:206] Beginning negotiation
0923 15:02:05.268880 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:02:05.268890 (+ 10us) server_negotiation.cc:407] Connection header received
0923 15:02:05.268915 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:02:05.268918 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:02:05.268955 (+ 37us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:02:05.269030 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:02:05.269253 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:05.270084 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:05.271983 (+ 1899us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:05.272467 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:05.273491 (+ 1024us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:05.273508 (+ 17us) server_negotiation.cc:1035] Waiting for connection context
0923 15:02:05.273530 (+ 22us) server_negotiation.cc:299] Negotiation successful
0923 15:02:05.273580 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105,"thread_start_us":68,"threads_started":1}
I20260923 15:02:05.274699 21050 tablet_bootstrap.cc:492] T a64d19ec71c4439ea3a00f04081ec3a3 P e9ef8b623efa4412ae2bfff361a0a8a8: Bootstrap starting.
I20260923 15:02:05.274706 20749 tablet_service.cc:1467] Processing CreateTablet for tablet a64d19ec71c4439ea3a00f04081ec3a3 (DEFAULT_TABLE table=test-table [id=206a01ebb8cc49fb92f7549443b8b158]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 15:02:05.274821 20749 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet a64d19ec71c4439ea3a00f04081ec3a3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:02:05.275185 21050 tablet_bootstrap.cc:654] T a64d19ec71c4439ea3a00f04081ec3a3 P e9ef8b623efa4412ae2bfff361a0a8a8: Neither blocks nor log segments found. Creating new log.
I20260923 15:02:05.275535 21051 tablet_bootstrap.cc:492] T a64d19ec71c4439ea3a00f04081ec3a3 P 430f9682c1fe49f7b9fc238400127887: Bootstrap starting.
I20260923 15:02:05.275859 21050 tablet_bootstrap.cc:492] T a64d19ec71c4439ea3a00f04081ec3a3 P e9ef8b623efa4412ae2bfff361a0a8a8: No bootstrap required, opened a new log
I20260923 15:02:05.275913 21051 tablet_bootstrap.cc:654] T a64d19ec71c4439ea3a00f04081ec3a3 P 430f9682c1fe49f7b9fc238400127887: Neither blocks nor log segments found. Creating new log.
I20260923 15:02:05.275921 21050 ts_tablet_manager.cc:1397] T a64d19ec71c4439ea3a00f04081ec3a3 P e9ef8b623efa4412ae2bfff361a0a8a8: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260923 15:02:05.276070 21050 raft_consensus.cc:348] T a64d19ec71c4439ea3a00f04081ec3a3 P e9ef8b623efa4412ae2bfff361a0a8a8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a32e8b9f779e4b9288aafcdb2e4a1c7a" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 42895 } } peers { permanent_uuid: "430f9682c1fe49f7b9fc238400127887" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 35617 } } peers { permanent_uuid: "e9ef8b623efa4412ae2bfff361a0a8a8" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 42411 } }
I20260923 15:02:05.276176 21050 raft_consensus.cc:374] T a64d19ec71c4439ea3a00f04081ec3a3 P e9ef8b623efa4412ae2bfff361a0a8a8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:02:05.276198 21050 raft_consensus.cc:729] T a64d19ec71c4439ea3a00f04081ec3a3 P e9ef8b623efa4412ae2bfff361a0a8a8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9ef8b623efa4412ae2bfff361a0a8a8, State: Initialized, Role: FOLLOWER
I20260923 15:02:05.276252 21050 consensus_queue.cc:260] T a64d19ec71c4439ea3a00f04081ec3a3 P e9ef8b623efa4412ae2bfff361a0a8a8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a32e8b9f779e4b9288aafcdb2e4a1c7a" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 42895 } } peers { permanent_uuid: "430f9682c1fe49f7b9fc238400127887" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 35617 } } peers { permanent_uuid: "e9ef8b623efa4412ae2bfff361a0a8a8" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 42411 } }
I20260923 15:02:05.276278 21054 tablet_bootstrap.cc:492] T a64d19ec71c4439ea3a00f04081ec3a3 P a32e8b9f779e4b9288aafcdb2e4a1c7a: Bootstrap starting.
I20260923 15:02:05.276405 21050 ts_tablet_manager.cc:1428] T a64d19ec71c4439ea3a00f04081ec3a3 P e9ef8b623efa4412ae2bfff361a0a8a8: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260923 15:02:05.276777 21054 tablet_bootstrap.cc:654] T a64d19ec71c4439ea3a00f04081ec3a3 P a32e8b9f779e4b9288aafcdb2e4a1c7a: Neither blocks nor log segments found. Creating new log.
I20260923 15:02:05.276924 21056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:05.276667 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:35255 (local address 127.13.75.129:34939)
0923 15:02:05.276821 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0923 15:02:05.276872 (+ 51us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.190:35255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":86,"threads_started":1}
I20260923 15:02:05.277333 21051 tablet_bootstrap.cc:492] T a64d19ec71c4439ea3a00f04081ec3a3 P 430f9682c1fe49f7b9fc238400127887: No bootstrap required, opened a new log
I20260923 15:02:05.277351 21054 tablet_bootstrap.cc:492] T a64d19ec71c4439ea3a00f04081ec3a3 P a32e8b9f779e4b9288aafcdb2e4a1c7a: No bootstrap required, opened a new log
I20260923 15:02:05.277377 21051 ts_tablet_manager.cc:1397] T a64d19ec71c4439ea3a00f04081ec3a3 P 430f9682c1fe49f7b9fc238400127887: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 15:02:05.277402 21054 ts_tablet_manager.cc:1397] T a64d19ec71c4439ea3a00f04081ec3a3 P a32e8b9f779e4b9288aafcdb2e4a1c7a: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260923 15:02:05.277474 21051 raft_consensus.cc:348] T a64d19ec71c4439ea3a00f04081ec3a3 P 430f9682c1fe49f7b9fc238400127887 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a32e8b9f779e4b9288aafcdb2e4a1c7a" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 42895 } } peers { permanent_uuid: "430f9682c1fe49f7b9fc238400127887" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 35617 } } peers { permanent_uuid: "e9ef8b623efa4412ae2bfff361a0a8a8" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 42411 } }
I20260923 15:02:05.277525 21051 raft_consensus.cc:374] T a64d19ec71c4439ea3a00f04081ec3a3 P 430f9682c1fe49f7b9fc238400127887 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:02:05.277539 21051 raft_consensus.cc:729] T a64d19ec71c4439ea3a00f04081ec3a3 P 430f9682c1fe49f7b9fc238400127887 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 430f9682c1fe49f7b9fc238400127887, State: Initialized, Role: FOLLOWER
I20260923 15:02:05.277535 21054 raft_consensus.cc:348] T a64d19ec71c4439ea3a00f04081ec3a3 P a32e8b9f779e4b9288aafcdb2e4a1c7a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a32e8b9f779e4b9288aafcdb2e4a1c7a" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 42895 } } peers { permanent_uuid: "430f9682c1fe49f7b9fc238400127887" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 35617 } } peers { permanent_uuid: "e9ef8b623efa4412ae2bfff361a0a8a8" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 42411 } }
I20260923 15:02:05.277601 21054 raft_consensus.cc:374] T a64d19ec71c4439ea3a00f04081ec3a3 P a32e8b9f779e4b9288aafcdb2e4a1c7a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:02:05.277601 21051 consensus_queue.cc:260] T a64d19ec71c4439ea3a00f04081ec3a3 P 430f9682c1fe49f7b9fc238400127887 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a32e8b9f779e4b9288aafcdb2e4a1c7a" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 42895 } } peers { permanent_uuid: "430f9682c1fe49f7b9fc238400127887" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 35617 } } peers { permanent_uuid: "e9ef8b623efa4412ae2bfff361a0a8a8" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 42411 } }
I20260923 15:02:05.277633 21054 raft_consensus.cc:729] T a64d19ec71c4439ea3a00f04081ec3a3 P a32e8b9f779e4b9288aafcdb2e4a1c7a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a32e8b9f779e4b9288aafcdb2e4a1c7a, State: Initialized, Role: FOLLOWER
I20260923 15:02:05.277688 21054 consensus_queue.cc:260] T a64d19ec71c4439ea3a00f04081ec3a3 P a32e8b9f779e4b9288aafcdb2e4a1c7a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a32e8b9f779e4b9288aafcdb2e4a1c7a" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 42895 } } peers { permanent_uuid: "430f9682c1fe49f7b9fc238400127887" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 35617 } } peers { permanent_uuid: "e9ef8b623efa4412ae2bfff361a0a8a8" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 42411 } }
I20260923 15:02:05.277762 21051 ts_tablet_manager.cc:1428] T a64d19ec71c4439ea3a00f04081ec3a3 P 430f9682c1fe49f7b9fc238400127887: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260923 15:02:05.277835 21054 ts_tablet_manager.cc:1428] T a64d19ec71c4439ea3a00f04081ec3a3 P a32e8b9f779e4b9288aafcdb2e4a1c7a: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260923 15:02:05.278450 20817 heartbeater.cc:503] Master 127.13.75.189:44251 was elected leader, sending a full tablet report...
I20260923 15:02:05.278622 20953 heartbeater.cc:503] Master 127.13.75.189:44251 was elected leader, sending a full tablet report...
I20260923 15:02:05.279004 21061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:05.278067 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:35255 (local address 127.13.75.130:39623)
0923 15:02:05.278732 (+ 665us) negotiation.cc:107] Waiting for socket to connect
0923 15:02:05.278963 (+ 231us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.190:35255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":545,"thread_start_us":98,"threads_started":1}
I20260923 15:02:05.279037 21060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:05.278011 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:35255 (local address 127.13.75.131:40061)
0923 15:02:05.278963 (+ 952us) negotiation.cc:107] Waiting for socket to connect
0923 15:02:05.278996 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.190:35255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":927,"thread_start_us":80,"threads_started":1}
I20260923 15:02:05.279346 21061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:05.279263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:35255 (local address 127.13.75.130:53695)
0923 15:02:05.279298 (+ 35us) negotiation.cc:107] Waiting for socket to connect
0923 15:02:05.279322 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.190:35255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
I20260923 15:02:05.279426 21060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:05.279333 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:35255 (local address 127.13.75.131:40939)
0923 15:02:05.279367 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0923 15:02:05.279404 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.190:35255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":18}
W20260923 15:02:05.290925 20954 tablet.cc:2369] T a64d19ec71c4439ea3a00f04081ec3a3 P 430f9682c1fe49f7b9fc238400127887: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 15:02:05.410874 21055 raft_consensus.cc:482] T a64d19ec71c4439ea3a00f04081ec3a3 P e9ef8b623efa4412ae2bfff361a0a8a8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 15:02:05.411399 21055 raft_consensus.cc:504] T a64d19ec71c4439ea3a00f04081ec3a3 P e9ef8b623efa4412ae2bfff361a0a8a8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a32e8b9f779e4b9288aafcdb2e4a1c7a" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 42895 } } peers { permanent_uuid: "430f9682c1fe49f7b9fc238400127887" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 35617 } } peers { permanent_uuid: "e9ef8b623efa4412ae2bfff361a0a8a8" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 42411 } }
I20260923 15:02:05.411814 21055 leader_election.cc:288] T a64d19ec71c4439ea3a00f04081ec3a3 P e9ef8b623efa4412ae2bfff361a0a8a8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers a32e8b9f779e4b9288aafcdb2e4a1c7a (127.13.75.130:42895), 430f9682c1fe49f7b9fc238400127887 (127.13.75.131:35617)
I20260923 15:02:05.415017 21045 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:05.412111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.129:51625 (local address 127.13.75.130:42895)
0923 15:02:05.412167 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0923 15:02:05.412169 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:02:05.412302 (+ 133us) server_negotiation.cc:407] Connection header received
0923 15:02:05.412374 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:02:05.412375 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:02:05.412386 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:02:05.412461 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:02:05.412616 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:05.413099 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:05.414218 (+ 1119us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:05.414608 (+ 390us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:05.414699 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:05.414711 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0923 15:02:05.414866 (+ 155us) server_negotiation.cc:299] Negotiation successful
0923 15:02:05.414940 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260923 15:02:05.415017 21062 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:05.412045 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.130:42895 (local address 127.13.75.129:51625)
0923 15:02:05.412223 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0923 15:02:05.412238 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0923 15:02:05.412316 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:02:05.412464 (+ 148us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:02:05.412465 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:02:05.412466 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:02:05.412536 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:05.412538 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:05.413193 (+ 655us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:05.413195 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:05.414112 (+ 917us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:05.414115 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:05.414700 (+ 585us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:05.414700 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:05.414768 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:05.414772 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 15:02:05.414855 (+ 83us) client_negotiation.cc:240] Negotiation successful
0923 15:02:05.414927 (+ 72us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"mutex_wait_us":52,"thread_start_us":108,"threads_started":1}
I20260923 15:02:05.415138 21046 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:05.412128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.129:51725 (local address 127.13.75.131:35617)
0923 15:02:05.412182 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0923 15:02:05.412184 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:02:05.412200 (+ 16us) server_negotiation.cc:407] Connection header received
0923 15:02:05.412289 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:02:05.412290 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:02:05.412300 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:02:05.412392 (+ 92us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:02:05.412578 (+ 186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:05.413110 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:05.414230 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:05.414634 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:05.414740 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:05.414750 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0923 15:02:05.414991 (+ 241us) server_negotiation.cc:299] Negotiation successful
0923 15:02:05.415039 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260923 15:02:05.415253 21056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:05.412030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.131:35617 (local address 127.13.75.129:51725)
0923 15:02:05.412123 (+ 93us) negotiation.cc:107] Waiting for socket to connect
0923 15:02:05.412138 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0923 15:02:05.412221 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:02:05.412406 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:02:05.412407 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:02:05.412409 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:02:05.412497 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:05.412499 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:05.413208 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:05.413209 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:05.414122 (+ 913us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:05.414125 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:05.414741 (+ 616us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:05.414742 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:05.414833 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:05.414837 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 15:02:05.414937 (+ 100us) client_negotiation.cc:240] Negotiation successful
0923 15:02:05.414992 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":67,"mutex_wait_us":44}
I20260923 15:02:05.415314 20769 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a64d19ec71c4439ea3a00f04081ec3a3" candidate_uuid: "e9ef8b623efa4412ae2bfff361a0a8a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a32e8b9f779e4b9288aafcdb2e4a1c7a" is_pre_election: true
I20260923 15:02:05.415511 20769 raft_consensus.cc:2454] T a64d19ec71c4439ea3a00f04081ec3a3 P a32e8b9f779e4b9288aafcdb2e4a1c7a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9ef8b623efa4412ae2bfff361a0a8a8 in term 0.
I20260923 15:02:05.415531 20905 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a64d19ec71c4439ea3a00f04081ec3a3" candidate_uuid: "e9ef8b623efa4412ae2bfff361a0a8a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "430f9682c1fe49f7b9fc238400127887" is_pre_election: true
I20260923 15:02:05.415633 20905 raft_consensus.cc:2454] T a64d19ec71c4439ea3a00f04081ec3a3 P 430f9682c1fe49f7b9fc238400127887 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e9ef8b623efa4412ae2bfff361a0a8a8 in term 0.
I20260923 15:02:05.415740 20565 leader_election.cc:302] T a64d19ec71c4439ea3a00f04081ec3a3 P e9ef8b623efa4412ae2bfff361a0a8a8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a32e8b9f779e4b9288aafcdb2e4a1c7a, e9ef8b623efa4412ae2bfff361a0a8a8; no voters:
I20260923 15:02:05.415946 21055 raft_consensus.cc:2781] T a64d19ec71c4439ea3a00f04081ec3a3 P e9ef8b623efa4412ae2bfff361a0a8a8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 15:02:05.416034 21055 raft_consensus.cc:482] T a64d19ec71c4439ea3a00f04081ec3a3 P e9ef8b623efa4412ae2bfff361a0a8a8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 15:02:05.416056 21055 raft_consensus.cc:3037] T a64d19ec71c4439ea3a00f04081ec3a3 P e9ef8b623efa4412ae2bfff361a0a8a8 [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:02:05.416806 21055 raft_consensus.cc:504] T a64d19ec71c4439ea3a00f04081ec3a3 P e9ef8b623efa4412ae2bfff361a0a8a8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a32e8b9f779e4b9288aafcdb2e4a1c7a" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 42895 } } peers { permanent_uuid: "430f9682c1fe49f7b9fc238400127887" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 35617 } } peers { permanent_uuid: "e9ef8b623efa4412ae2bfff361a0a8a8" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 42411 } }
I20260923 15:02:05.416958 21055 leader_election.cc:288] T a64d19ec71c4439ea3a00f04081ec3a3 P e9ef8b623efa4412ae2bfff361a0a8a8 [CANDIDATE]: Term 1 election: Requested vote from peers a32e8b9f779e4b9288aafcdb2e4a1c7a (127.13.75.130:42895), 430f9682c1fe49f7b9fc238400127887 (127.13.75.131:35617)
I20260923 15:02:05.417138 20905 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a64d19ec71c4439ea3a00f04081ec3a3" candidate_uuid: "e9ef8b623efa4412ae2bfff361a0a8a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "430f9682c1fe49f7b9fc238400127887"
I20260923 15:02:05.417138 20769 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "a64d19ec71c4439ea3a00f04081ec3a3" candidate_uuid: "e9ef8b623efa4412ae2bfff361a0a8a8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a32e8b9f779e4b9288aafcdb2e4a1c7a"
I20260923 15:02:05.417239 20905 raft_consensus.cc:3037] T a64d19ec71c4439ea3a00f04081ec3a3 P 430f9682c1fe49f7b9fc238400127887 [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:02:05.417260 20769 raft_consensus.cc:3037] T a64d19ec71c4439ea3a00f04081ec3a3 P a32e8b9f779e4b9288aafcdb2e4a1c7a [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:02:05.417892 20905 raft_consensus.cc:2454] T a64d19ec71c4439ea3a00f04081ec3a3 P 430f9682c1fe49f7b9fc238400127887 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9ef8b623efa4412ae2bfff361a0a8a8 in term 1.
I20260923 15:02:05.418056 20769 raft_consensus.cc:2454] T a64d19ec71c4439ea3a00f04081ec3a3 P a32e8b9f779e4b9288aafcdb2e4a1c7a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9ef8b623efa4412ae2bfff361a0a8a8 in term 1.
I20260923 15:02:05.418075 20567 leader_election.cc:302] T a64d19ec71c4439ea3a00f04081ec3a3 P e9ef8b623efa4412ae2bfff361a0a8a8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 430f9682c1fe49f7b9fc238400127887, e9ef8b623efa4412ae2bfff361a0a8a8; no voters:
I20260923 15:02:05.418174 21055 raft_consensus.cc:2781] T a64d19ec71c4439ea3a00f04081ec3a3 P e9ef8b623efa4412ae2bfff361a0a8a8 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 15:02:05.418327 21055 raft_consensus.cc:686] T a64d19ec71c4439ea3a00f04081ec3a3 P e9ef8b623efa4412ae2bfff361a0a8a8 [term 1 LEADER]: Becoming Leader. State: Replica: e9ef8b623efa4412ae2bfff361a0a8a8, State: Running, Role: LEADER
I20260923 15:02:05.418416 21055 consensus_queue.cc:237] T a64d19ec71c4439ea3a00f04081ec3a3 P e9ef8b623efa4412ae2bfff361a0a8a8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a32e8b9f779e4b9288aafcdb2e4a1c7a" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 42895 } } peers { permanent_uuid: "430f9682c1fe49f7b9fc238400127887" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 35617 } } peers { permanent_uuid: "e9ef8b623efa4412ae2bfff361a0a8a8" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 42411 } }
I20260923 15:02:05.418649 21056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:05.418543 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:35255 (local address 127.13.75.129:49759)
0923 15:02:05.418581 (+ 38us) negotiation.cc:107] Waiting for socket to connect
0923 15:02:05.418614 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.190:35255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
I20260923 15:02:05.418797 20680 heartbeater.cc:503] Master 127.13.75.189:44251 was elected leader, sending a full tablet report...
I20260923 15:02:05.419237 20310 catalog_manager.cc:5495] T a64d19ec71c4439ea3a00f04081ec3a3 P e9ef8b623efa4412ae2bfff361a0a8a8 reported cstate change: term changed from 0 to 1, leader changed from <none> to e9ef8b623efa4412ae2bfff361a0a8a8 (127.13.75.129). New cstate: current_term: 1 leader_uuid: "e9ef8b623efa4412ae2bfff361a0a8a8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "a32e8b9f779e4b9288aafcdb2e4a1c7a" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 42895 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "430f9682c1fe49f7b9fc238400127887" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 35617 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e9ef8b623efa4412ae2bfff361a0a8a8" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 42411 } health_report { overall_health: HEALTHY } } }
I20260923 15:02:05.457617 21047 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:05.454781 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50018 (local address 127.13.75.129:42411)
0923 15:02:05.454847 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0923 15:02:05.454849 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:02:05.454859 (+ 10us) server_negotiation.cc:407] Connection header received
0923 15:02:05.454968 (+ 109us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:02:05.454969 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:02:05.454997 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:02:05.455061 (+ 64us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 15:02:05.455227 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:05.455747 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:05.456549 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:05.456774 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:05.456836 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:05.457062 (+ 226us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 15:02:05.457262 (+ 200us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 15:02:05.457347 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0923 15:02:05.457462 (+ 115us) server_negotiation.cc:299] Negotiation successful
0923 15:02:05.457491 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260923 15:02:05.459091 20905 raft_consensus.cc:1261] T a64d19ec71c4439ea3a00f04081ec3a3 P 430f9682c1fe49f7b9fc238400127887 [term 1 FOLLOWER]: Refusing update from remote peer e9ef8b623efa4412ae2bfff361a0a8a8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 15:02:05.459170 20769 raft_consensus.cc:1261] T a64d19ec71c4439ea3a00f04081ec3a3 P a32e8b9f779e4b9288aafcdb2e4a1c7a [term 1 FOLLOWER]: Refusing update from remote peer e9ef8b623efa4412ae2bfff361a0a8a8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 15:02:05.459436 21055 consensus_queue.cc:1035] T a64d19ec71c4439ea3a00f04081ec3a3 P e9ef8b623efa4412ae2bfff361a0a8a8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "430f9682c1fe49f7b9fc238400127887" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 35617 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 15:02:05.459633 21063 consensus_queue.cc:1035] T a64d19ec71c4439ea3a00f04081ec3a3 P e9ef8b623efa4412ae2bfff361a0a8a8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "a32e8b9f779e4b9288aafcdb2e4a1c7a" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 42895 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 15:02:05.460289 21060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:05.460058 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:35255 (local address 127.13.75.131:51579)
0923 15:02:05.460214 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0923 15:02:05.460252 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.190:35255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133}
I20260923 15:02:05.461014 21061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:05.460901 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:35255 (local address 127.13.75.130:56787)
0923 15:02:05.460943 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0923 15:02:05.460980 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.190:35255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":23}
I20260923 15:02:05.462308 21061 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:05.462197 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:35255 (local address 127.13.75.130:45467)
0923 15:02:05.462237 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0923 15:02:05.462273 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.190:35255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":21}
I20260923 15:02:05.463006 21056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:05.462772 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:35255 (local address 127.13.75.129:41613)
0923 15:02:05.462936 (+ 164us) negotiation.cc:107] Waiting for socket to connect
0923 15:02:05.462973 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.190:35255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":117}
I20260923 15:02:05.463033 21060 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:05.462335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:35255 (local address 127.13.75.131:49917)
0923 15:02:05.462962 (+ 627us) negotiation.cc:107] Waiting for socket to connect
0923 15:02:05.463000 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.190:35255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":607}
I20260923 15:02:05.463512 21056 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:05.463399 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:35255 (local address 127.13.75.129:58311)
0923 15:02:05.463450 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0923 15:02:05.463482 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.190:35255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260923 15:02:05.465041 20309 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39198:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 15:02:05.465143 20309 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39198:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 15:02:05.466650 20309 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b8c37de502e542aab23126e30c0e3176: Sending DeleteTablet for 3 replicas of tablet a64d19ec71c4439ea3a00f04081ec3a3
I20260923 15:02:05.467013 20749 tablet_service.cc:1514] Processing DeleteTablet for tablet a64d19ec71c4439ea3a00f04081ec3a3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 15:02:05 UTC) from {username='oryx', principal='oryx/127.13.75.189@KRBTEST.COM'} at 127.0.0.1:37420
I20260923 15:02:05.467033 20885 tablet_service.cc:1514] Processing DeleteTablet for tablet a64d19ec71c4439ea3a00f04081ec3a3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 15:02:05 UTC) from {username='oryx', principal='oryx/127.13.75.189@KRBTEST.COM'} at 127.0.0.1:39008
I20260923 15:02:05.467211 20612 tablet_service.cc:1514] Processing DeleteTablet for tablet a64d19ec71c4439ea3a00f04081ec3a3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 15:02:05 UTC) from {username='oryx', principal='oryx/127.13.75.189@KRBTEST.COM'} at 127.0.0.1:50008
I20260923 15:02:05.467458 21077 tablet_replica.cc:330] T a64d19ec71c4439ea3a00f04081ec3a3 P a32e8b9f779e4b9288aafcdb2e4a1c7a: stopping tablet replica
I20260923 15:02:05.467535 21077 raft_consensus.cc:2229] T a64d19ec71c4439ea3a00f04081ec3a3 P a32e8b9f779e4b9288aafcdb2e4a1c7a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 15:02:05.467597 21077 raft_consensus.cc:2258] T a64d19ec71c4439ea3a00f04081ec3a3 P a32e8b9f779e4b9288aafcdb2e4a1c7a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 15:02:05.467633 13614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKC4soy/build/release/bin/kudu with pid 20458
I20260923 15:02:05.468537 21077 ts_tablet_manager.cc:1905] T a64d19ec71c4439ea3a00f04081ec3a3 P a32e8b9f779e4b9288aafcdb2e4a1c7a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 15:02:05.467350 21076 tablet_replica.cc:330] T a64d19ec71c4439ea3a00f04081ec3a3 P 430f9682c1fe49f7b9fc238400127887: stopping tablet replica
I20260923 15:02:05.469377 21076 raft_consensus.cc:2229] T a64d19ec71c4439ea3a00f04081ec3a3 P 430f9682c1fe49f7b9fc238400127887 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 15:02:05.469441 21076 raft_consensus.cc:2258] T a64d19ec71c4439ea3a00f04081ec3a3 P 430f9682c1fe49f7b9fc238400127887 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 15:02:05.469786 21076 ts_tablet_manager.cc:1905] T a64d19ec71c4439ea3a00f04081ec3a3 P 430f9682c1fe49f7b9fc238400127887: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 15:02:05.470863 21077 ts_tablet_manager.cc:1918] T a64d19ec71c4439ea3a00f04081ec3a3 P a32e8b9f779e4b9288aafcdb2e4a1c7a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 15:02:05.470971 21077 log.cc:1196] T a64d19ec71c4439ea3a00f04081ec3a3 P a32e8b9f779e4b9288aafcdb2e4a1c7a: Deleting WAL directory at /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/minicluster-data/ts-1/wal/wals/a64d19ec71c4439ea3a00f04081ec3a3
I20260923 15:02:05.471267 21077 ts_tablet_manager.cc:1939] T a64d19ec71c4439ea3a00f04081ec3a3 P a32e8b9f779e4b9288aafcdb2e4a1c7a: Deleting consensus metadata
I20260923 15:02:05.471638 20294 catalog_manager.cc:4841] TS a32e8b9f779e4b9288aafcdb2e4a1c7a (127.13.75.130:42895): tablet a64d19ec71c4439ea3a00f04081ec3a3 (table test-table [id=206a01ebb8cc49fb92f7549443b8b158]) successfully deleted
I20260923 15:02:05.471814 21076 ts_tablet_manager.cc:1918] T a64d19ec71c4439ea3a00f04081ec3a3 P 430f9682c1fe49f7b9fc238400127887: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 15:02:05.471884 21076 log.cc:1196] T a64d19ec71c4439ea3a00f04081ec3a3 P 430f9682c1fe49f7b9fc238400127887: Deleting WAL directory at /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipalMultipleMaster.1790175691491353-13614-0/minicluster-data/ts-2/wal/wals/a64d19ec71c4439ea3a00f04081ec3a3
I20260923 15:02:05.472101 21076 ts_tablet_manager.cc:1939] T a64d19ec71c4439ea3a00f04081ec3a3 P 430f9682c1fe49f7b9fc238400127887: Deleting consensus metadata
I20260923 15:02:05.472644 20296 catalog_manager.cc:4841] TS 430f9682c1fe49f7b9fc238400127887 (127.13.75.131:35617): tablet a64d19ec71c4439ea3a00f04081ec3a3 (table test-table [id=206a01ebb8cc49fb92f7549443b8b158]) successfully deleted
W20260923 15:02:05.474506 21048 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 15:02:05.474278 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.129:42411 (local address 127.0.0.1:50020)
0923 15:02:05.474376 (+ 98us) negotiation.cc:107] Waiting for socket to connect
0923 15:02:05.474413 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.129:42411: connect: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":62}
I20260923 15:02:05.474611 13614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKC4soy/build/release/bin/kudu with pid 20686
W20260923 15:02:05.474702 20294 catalog_manager.cc:4584] TS e9ef8b623efa4412ae2bfff361a0a8a8 (127.13.75.129:42411): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet a64d19ec71c4439ea3a00f04081ec3a3: Network error: Client connection negotiation failed: client connection to 127.13.75.129:42411: connect: Connection reset by peer (error 104)
I20260923 15:02:05.483058 13614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKC4soy/build/release/bin/kudu with pid 20822
I20260923 15:02:05.488211 13614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKC4soy/build/release/bin/kudu with pid 20174
I20260923 15:02:05.493427 13614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKC4soy/build/release/bin/kudu with pid 20364
2026-09-23T15:02:05Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipalMultipleMaster (8533 ms)
[ RUN ] SecurityITest.TestNonDefaultPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175691491353-13614-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 15:02:05 dist-test-slave-cnb0 krb5kdc[21081](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 15:02:05 dist-test-slave-cnb0 krb5kdc[21081](info): set up 2 sockets
Sep 23 15:02:05 dist-test-slave-cnb0 krb5kdc[21081](info): commencing operation
krb5kdc: starting...
W20260923 15:02:06.572683 13614 mini_kdc.cc:121] Time spent starting KDC: real 1.057s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175691491353-13614-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175691491353-13614-0/krb5kdc/test-user.keytab.
Sep 23 15:02:06 dist-test-slave-cnb0 krb5kdc[21081](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175726, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T15:02:06Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T15:02:06Z Disabled control of system clock
WARNING: no policy specified for oryx/127.13.75.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.13.75.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.13.75.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175691491353-13614-0/krb5kdc/oryx_127.13.75.190.keytab.
Entry for principal oryx/127.13.75.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175691491353-13614-0/krb5kdc/oryx_127.13.75.190.keytab.
I20260923 15:02:06.699157 13614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKC4soy/build/release/bin/kudu
/tmp/dist-test-taskKC4soy/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175691491353-13614-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175691491353-13614-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175691491353-13614-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175691491353-13614-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.75.190:40515
--webserver_interface=127.13.75.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175691491353-13614-0/krb5kdc/oryx_127.13.75.190.keytab
--principal=oryx/127.13.75.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.75.148:33291
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.75.190:40515
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--txn_manager_enabled=true with env {KRB5CCNAME=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175691491353-13614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175691491353-13614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175691491353-13614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:02:06.784237 21094 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:02:06.784490 21094 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:02:06.784545 21094 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:02:06.785773 21094 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 15:02:06.785845 21094 flags.cc:425] Enabled experimental flag: --txn_manager_enabled=true
W20260923 15:02:06.785871 21094 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:02:06.785904 21094 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:02:06.785931 21094 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 15:02:06.785961 21094 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 15:02:06.787658 21094 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.75.148:33291
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175691491353-13614-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175691491353-13614-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.75.190:40515
--txn_manager_enabled=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175691491353-13614-0/krb5kdc/oryx_127.13.75.190.keytab
--principal=oryx/127.13.75.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.75.190:40515
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175691491353-13614-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.75.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21094
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175691491353-13614-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:02:06.787952 21094 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:02:06.788203 21094 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:02:06.794348 21100 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:02:06.794333 21102 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:02:06.794333 21099 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:02:06.794687 21094 server_base.cc:1034] running on GCE node
I20260923 15:02:06.795090 21094 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:02:06.795449 21094 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:02:06.796649 21094 hybrid_clock.cc:648] HybridClock initialized: now 1790175726796630 us; error 44 us; skew 500 ppm
Sep 23 15:02:06 dist-test-slave-cnb0 krb5kdc[21081](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175726, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.75.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:02:06.799521 21094 init.cc:377] Logged in from keytab as oryx/127.13.75.190@KRBTEST.COM (short username oryx)
I20260923 15:02:06.800068 21094 webserver.cc:460] Webserver started at http://127.13.75.190:44053/ using document root <none> and password file <none>
I20260923 15:02:06.800338 21094 fs_manager.cc:359] Metadata directory not provided
I20260923 15:02:06.800410 21094 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:02:06.800580 21094 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:02:06.801700 21094 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175691491353-13614-0/minicluster-data/master-0/data/instance:
uuid: "385ef6dbf6774aa09aff37a6dcf6afd0"
format_stamp: "Formatted at 2026-09-23 15:02:06 on dist-test-slave-cnb0"
server_key: "d1d99293894f1c42c60f6bc360008a65"
server_key_iv: "779f983a6aeda2a64db02a9aa858d3fb"
server_key_version: "encryptionkey@0"
I20260923 15:02:06.802078 21094 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175691491353-13614-0/minicluster-data/master-0/wal/instance:
uuid: "385ef6dbf6774aa09aff37a6dcf6afd0"
format_stamp: "Formatted at 2026-09-23 15:02:06 on dist-test-slave-cnb0"
server_key: "d1d99293894f1c42c60f6bc360008a65"
server_key_iv: "779f983a6aeda2a64db02a9aa858d3fb"
server_key_version: "encryptionkey@0"
I20260923 15:02:06.803639 21094 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 15:02:06.804520 21109 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:02:06.804807 21094 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:02:06.804962 21094 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175691491353-13614-0/minicluster-data/master-0/data,/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175691491353-13614-0/minicluster-data/master-0/wal
uuid: "385ef6dbf6774aa09aff37a6dcf6afd0"
format_stamp: "Formatted at 2026-09-23 15:02:06 on dist-test-slave-cnb0"
server_key: "d1d99293894f1c42c60f6bc360008a65"
server_key_iv: "779f983a6aeda2a64db02a9aa858d3fb"
server_key_version: "encryptionkey@0"
I20260923 15:02:06.805055 21094 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175691491353-13614-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175691491353-13614-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175691491353-13614-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:02:06.815414 21094 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:02:06.818113 21094 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:02:06.818320 21094 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:02:06.823596 21094 rpc_server.cc:307] RPC server started. Bound to: 127.13.75.190:40515
I20260923 15:02:06.823684 21171 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.75.190:40515 every 8 connection(s)
I20260923 15:02:06.824044 21094 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175691491353-13614-0/minicluster-data/master-0/data/info.pb
I20260923 15:02:06.824680 13614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKC4soy/build/release/bin/kudu as pid 21094
I20260923 15:02:06.824877 13614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175691491353-13614-0/minicluster-data/master-0/wal/instance
I20260923 15:02:06.825105 21172 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:02:06.825147 13614 external_mini_cluster.cc:1428] Setting key fbf3b8b9a3653668ec2541e94a2aa04f
I20260923 15:02:06.828128 21172 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 385ef6dbf6774aa09aff37a6dcf6afd0: Bootstrap starting.
I20260923 15:02:06.828817 21172 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 385ef6dbf6774aa09aff37a6dcf6afd0: Neither blocks nor log segments found. Creating new log.
I20260923 15:02:06.829116 21172 log.cc:824] T 00000000000000000000000000000000 P 385ef6dbf6774aa09aff37a6dcf6afd0: Log is configured to *not* fsync() on all Append() calls
I20260923 15:02:06.829814 21172 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 385ef6dbf6774aa09aff37a6dcf6afd0: No bootstrap required, opened a new log
Sep 23 15:02:06 dist-test-slave-cnb0 krb5kdc[21081](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175726, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.13.75.190@KRBTEST.COM
I20260923 15:02:06.832614 21172 raft_consensus.cc:348] T 00000000000000000000000000000000 P 385ef6dbf6774aa09aff37a6dcf6afd0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "385ef6dbf6774aa09aff37a6dcf6afd0" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 40515 } }
I20260923 15:02:06.832842 21172 raft_consensus.cc:374] T 00000000000000000000000000000000 P 385ef6dbf6774aa09aff37a6dcf6afd0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:02:06.832873 21172 raft_consensus.cc:729] T 00000000000000000000000000000000 P 385ef6dbf6774aa09aff37a6dcf6afd0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 385ef6dbf6774aa09aff37a6dcf6afd0, State: Initialized, Role: FOLLOWER
I20260923 15:02:06.833031 21172 consensus_queue.cc:260] T 00000000000000000000000000000000 P 385ef6dbf6774aa09aff37a6dcf6afd0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "385ef6dbf6774aa09aff37a6dcf6afd0" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 40515 } }
I20260923 15:02:06.833112 21172 raft_consensus.cc:388] T 00000000000000000000000000000000 P 385ef6dbf6774aa09aff37a6dcf6afd0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 15:02:06.833137 21172 raft_consensus.cc:482] T 00000000000000000000000000000000 P 385ef6dbf6774aa09aff37a6dcf6afd0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 15:02:06.833205 21172 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 385ef6dbf6774aa09aff37a6dcf6afd0 [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:02:06.833922 21172 raft_consensus.cc:504] T 00000000000000000000000000000000 P 385ef6dbf6774aa09aff37a6dcf6afd0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "385ef6dbf6774aa09aff37a6dcf6afd0" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 40515 } }
I20260923 15:02:06.834034 21172 leader_election.cc:302] T 00000000000000000000000000000000 P 385ef6dbf6774aa09aff37a6dcf6afd0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 385ef6dbf6774aa09aff37a6dcf6afd0; no voters:
I20260923 15:02:06.834259 21172 leader_election.cc:288] T 00000000000000000000000000000000 P 385ef6dbf6774aa09aff37a6dcf6afd0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 15:02:06.834291 21177 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 385ef6dbf6774aa09aff37a6dcf6afd0 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 15:02:06.834482 21177 raft_consensus.cc:686] T 00000000000000000000000000000000 P 385ef6dbf6774aa09aff37a6dcf6afd0 [term 1 LEADER]: Becoming Leader. State: Replica: 385ef6dbf6774aa09aff37a6dcf6afd0, State: Running, Role: LEADER
I20260923 15:02:06.834651 21172 sys_catalog.cc:564] T 00000000000000000000000000000000 P 385ef6dbf6774aa09aff37a6dcf6afd0 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 15:02:06.834630 21177 consensus_queue.cc:237] T 00000000000000000000000000000000 P 385ef6dbf6774aa09aff37a6dcf6afd0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "385ef6dbf6774aa09aff37a6dcf6afd0" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 40515 } }
I20260923 15:02:06.835107 21177 sys_catalog.cc:455] T 00000000000000000000000000000000 P 385ef6dbf6774aa09aff37a6dcf6afd0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 385ef6dbf6774aa09aff37a6dcf6afd0. Latest consensus state: current_term: 1 leader_uuid: "385ef6dbf6774aa09aff37a6dcf6afd0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "385ef6dbf6774aa09aff37a6dcf6afd0" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 40515 } } }
I20260923 15:02:06.835114 21179 sys_catalog.cc:455] T 00000000000000000000000000000000 P 385ef6dbf6774aa09aff37a6dcf6afd0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "385ef6dbf6774aa09aff37a6dcf6afd0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "385ef6dbf6774aa09aff37a6dcf6afd0" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 40515 } } }
I20260923 15:02:06.835311 21177 sys_catalog.cc:458] T 00000000000000000000000000000000 P 385ef6dbf6774aa09aff37a6dcf6afd0 [sys.catalog]: This master's current role is: LEADER
I20260923 15:02:06.835328 21179 sys_catalog.cc:458] T 00000000000000000000000000000000 P 385ef6dbf6774aa09aff37a6dcf6afd0 [sys.catalog]: This master's current role is: LEADER
I20260923 15:02:06.835561 21186 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 15:02:06.836144 21186 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 15:02:06.837664 21175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:06.825756 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35120 (local address 127.13.75.190:40515)
0923 15:02:06.825952 (+ 196us) server_negotiation.cc:206] Beginning negotiation
0923 15:02:06.825962 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0923 15:02:06.825995 (+ 33us) server_negotiation.cc:407] Connection header received
0923 15:02:06.826599 (+ 604us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:02:06.826607 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:02:06.826660 (+ 53us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:02:06.826813 (+ 153us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:02:06.827495 (+ 682us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:06.828472 (+ 977us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:06.829360 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:06.829793 (+ 433us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:06.829917 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:06.833264 (+ 3347us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:02:06.833270 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:02:06.833270 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:02:06.833284 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:02:06.835657 (+ 2373us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:02:06.836144 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:02:06.836147 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:02:06.836148 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:02:06.836291 (+ 143us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:02:06.836552 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:02:06.836553 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:02:06.836553 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:02:06.836879 (+ 326us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:02:06.837225 (+ 346us) server_negotiation.cc:1035] Waiting for connection context
0923 15:02:06.837306 (+ 81us) server_negotiation.cc:299] Negotiation successful
0923 15:02:06.837457 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":131,"thread_start_us":78,"threads_started":1}
I20260923 15:02:06.843614 21186 catalog_manager.cc:1347] Generated new cluster ID: 262968426d994630aa1dbd453f43759f
I20260923 15:02:06.843709 21186 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 15:02:06.860783 21186 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 15:02:06.861390 21186 catalog_manager.cc:1504] Loading token signing keys...
I20260923 15:02:06.869424 21186 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 385ef6dbf6774aa09aff37a6dcf6afd0: Generated new TSK 0
I20260923 15:02:06.869692 21186 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for oryx/127.13.75.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.13.75.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.13.75.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175691491353-13614-0/krb5kdc/oryx_127.13.75.129.keytab.
Entry for principal oryx/127.13.75.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175691491353-13614-0/krb5kdc/oryx_127.13.75.129.keytab.
I20260923 15:02:06.908083 13614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKC4soy/build/release/bin/kudu
/tmp/dist-test-taskKC4soy/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175691491353-13614-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175691491353-13614-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175691491353-13614-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175691491353-13614-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.75.129:0
--local_ip_for_outbound_sockets=127.13.75.129
--webserver_interface=127.13.75.129
--webserver_port=0
--tserver_master_addrs=127.13.75.190:40515
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175691491353-13614-0/krb5kdc/oryx_127.13.75.129.keytab
--principal=oryx/127.13.75.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.75.148:33291
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175691491353-13614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175691491353-13614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175691491353-13614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:02:06.992888 21198 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:02:06.993162 21198 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:02:06.993209 21198 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:02:06.994518 21198 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:02:06.994625 21198 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:02:06.995050 21198 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260923 15:02:06.995146 21198 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.75.129
I20260923 15:02:06.997503 21198 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.75.148:33291
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175691491353-13614-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175691491353-13614-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175691491353-13614-0/krb5kdc/oryx_127.13.75.129.keytab
--principal=oryx/127.13.75.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.75.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175691491353-13614-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.75.129
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.13.75.190:40515
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21198
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.75.129
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175691491353-13614-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:02:06.997867 21198 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:02:06.998164 21198 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:02:07.004276 21204 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:02:07.004470 21203 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:02:07.004609 21206 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:02:07.004581 21198 server_base.cc:1034] running on GCE node
I20260923 15:02:07.005059 21198 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:02:07.005388 21198 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:02:07.006610 21198 hybrid_clock.cc:648] HybridClock initialized: now 1790175727006601 us; error 65 us; skew 500 ppm
Sep 23 15:02:07 dist-test-slave-cnb0 krb5kdc[21081](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175727, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.75.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:02:07.010047 21198 init.cc:377] Logged in from keytab as oryx/127.13.75.129@KRBTEST.COM (short username oryx)
I20260923 15:02:07.010625 21198 webserver.cc:460] Webserver started at http://127.13.75.129:34191/ using document root <none> and password file <none>
I20260923 15:02:07.010977 21198 fs_manager.cc:359] Metadata directory not provided
I20260923 15:02:07.011032 21198 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:02:07.011217 21198 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:02:07.012359 21198 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175691491353-13614-0/minicluster-data/ts-0/data/instance:
uuid: "f2f18bf473b04a23a122a756e8f646a3"
format_stamp: "Formatted at 2026-09-23 15:02:07 on dist-test-slave-cnb0"
server_key: "11b2fe90d4ec61045e77429dfe2fd39e"
server_key_iv: "6a295d755487e1a29aa926eb605902a4"
server_key_version: "encryptionkey@0"
I20260923 15:02:07.012742 21198 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175691491353-13614-0/minicluster-data/ts-0/wal/instance:
uuid: "f2f18bf473b04a23a122a756e8f646a3"
format_stamp: "Formatted at 2026-09-23 15:02:07 on dist-test-slave-cnb0"
server_key: "11b2fe90d4ec61045e77429dfe2fd39e"
server_key_iv: "6a295d755487e1a29aa926eb605902a4"
server_key_version: "encryptionkey@0"
I20260923 15:02:07.014268 21198 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 15:02:07.015501 21213 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:02:07.015782 21198 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260923 15:02:07.015954 21198 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175691491353-13614-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175691491353-13614-0/minicluster-data/ts-0/wal
uuid: "f2f18bf473b04a23a122a756e8f646a3"
format_stamp: "Formatted at 2026-09-23 15:02:07 on dist-test-slave-cnb0"
server_key: "11b2fe90d4ec61045e77429dfe2fd39e"
server_key_iv: "6a295d755487e1a29aa926eb605902a4"
server_key_version: "encryptionkey@0"
I20260923 15:02:07.016052 21198 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175691491353-13614-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175691491353-13614-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175691491353-13614-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:02:07.035766 21198 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:02:07.038631 21198 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:02:07.038846 21198 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:02:07.039645 21198 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 15:02:07.039710 21198 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 15:02:07.039754 21198 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 15:02:07.039803 21198 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 23 15:02:07 dist-test-slave-cnb0 krb5kdc[21081](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175727, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.75.129@KRBTEST.COM for oryx/127.13.75.190@KRBTEST.COM
I20260923 15:02:07.048213 21198 rpc_server.cc:307] RPC server started. Bound to: 127.13.75.129:38767
I20260923 15:02:07.048291 21328 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.75.129:38767 every 8 connection(s)
I20260923 15:02:07.048730 21198 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175691491353-13614-0/minicluster-data/ts-0/data/info.pb
I20260923 15:02:07.051366 21222 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:07.040089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:40515 (local address 127.13.75.129:55295)
0923 15:02:07.040273 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0923 15:02:07.040315 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0923 15:02:07.040833 (+ 518us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:02:07.041161 (+ 328us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:02:07.041166 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:02:07.041516 (+ 350us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:02:07.041838 (+ 322us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:07.041844 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:07.042963 (+ 1119us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:07.042964 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:07.043867 (+ 903us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:07.043870 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:07.044458 (+ 588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:07.044460 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:07.044660 (+ 200us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:07.045322 (+ 662us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:02:07.045333 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:02:07.047274 (+ 1941us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:02:07.049564 (+ 2290us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:02:07.049571 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:02:07.049574 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:02:07.049839 (+ 265us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:02:07.050095 (+ 256us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:02:07.050096 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:02:07.050096 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:02:07.050210 (+ 114us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:02:07.050643 (+ 433us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:02:07.050649 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:02:07.050782 (+ 133us) client_negotiation.cc:769] Sending connection context
0923 15:02:07.050966 (+ 184us) client_negotiation.cc:240] Negotiation successful
0923 15:02:07.051189 (+ 223us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":88,"threads_started":1}
I20260923 15:02:07.051263 21175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:07.040083 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.129:55295 (local address 127.13.75.190:40515)
0923 15:02:07.040156 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0923 15:02:07.040159 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:02:07.040428 (+ 269us) server_negotiation.cc:407] Connection header received
0923 15:02:07.040949 (+ 521us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:02:07.040951 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:02:07.040967 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:02:07.041073 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:02:07.041965 (+ 892us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:07.042796 (+ 831us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:07.043982 (+ 1186us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:07.044349 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:07.044449 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:07.047398 (+ 2949us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:02:07.047400 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:02:07.047401 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:02:07.047413 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:02:07.049449 (+ 2036us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:02:07.049944 (+ 495us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:02:07.049945 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:02:07.049946 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:02:07.050007 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:02:07.050334 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:02:07.050335 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:02:07.050336 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:02:07.050519 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:02:07.050597 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0923 15:02:07.051086 (+ 489us) server_negotiation.cc:299] Negotiation successful
0923 15:02:07.051179 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260923 15:02:07.052004 21329 heartbeater.cc:347] Connected to a master server at 127.13.75.190:40515
I20260923 15:02:07.052086 21329 heartbeater.cc:464] Registering TS with master...
I20260923 15:02:07.052274 21329 heartbeater.cc:511] Master 127.13.75.190:40515 requested a full tablet report, sending...
I20260923 15:02:07.052858 21126 ts_manager.cc:194] Registered new tserver with Master: f2f18bf473b04a23a122a756e8f646a3 (127.13.75.129:38767)
I20260923 15:02:07.053617 21126 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.13.75.129@KRBTEST.COM'} at 127.13.75.129:55295
I20260923 15:02:07.054538 13614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKC4soy/build/release/bin/kudu as pid 21198
I20260923 15:02:07.054646 13614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175691491353-13614-0/minicluster-data/ts-0/wal/instance
I20260923 15:02:07.054841 13614 external_mini_cluster.cc:1428] Setting key 3b98d4bafec64b2e745d68b7d405f9b4
I20260923 15:02:07.059836 21335 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:07.053115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:40515 (local address 127.13.75.129:53217)
0923 15:02:07.053253 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0923 15:02:07.053263 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 15:02:07.053328 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:02:07.053463 (+ 135us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:02:07.053463 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:02:07.053649 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:02:07.053731 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:07.053733 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:07.054371 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:07.054372 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:07.055135 (+ 763us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:07.055138 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:07.055505 (+ 367us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:07.055506 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:07.055571 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:07.056067 (+ 496us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:02:07.056073 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:02:07.056496 (+ 423us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:02:07.058664 (+ 2168us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:02:07.058665 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:02:07.058665 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:02:07.058843 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:02:07.059152 (+ 309us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:02:07.059153 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:02:07.059153 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:02:07.059211 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:02:07.059554 (+ 343us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:02:07.059555 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:02:07.059596 (+ 41us) client_negotiation.cc:769] Sending connection context
0923 15:02:07.059647 (+ 51us) client_negotiation.cc:240] Negotiation successful
0923 15:02:07.059738 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":80,"threads_started":1}
I20260923 15:02:07.059967 21175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:07.053186 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.129:53217 (local address 127.13.75.190:40515)
0923 15:02:07.053230 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0923 15:02:07.053232 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:02:07.053326 (+ 94us) server_negotiation.cc:407] Connection header received
0923 15:02:07.053389 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:02:07.053390 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:02:07.053401 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:02:07.053456 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:02:07.053793 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:07.054272 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:07.055198 (+ 926us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:07.055418 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:07.055503 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:07.056602 (+ 1099us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:02:07.056603 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:02:07.056604 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:02:07.056610 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:02:07.058557 (+ 1947us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:02:07.059008 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:02:07.059009 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:02:07.059010 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:02:07.059079 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:02:07.059319 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:02:07.059320 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:02:07.059320 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:02:07.059466 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:02:07.059507 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0923 15:02:07.059708 (+ 201us) server_negotiation.cc:299] Negotiation successful
0923 15:02:07.059814 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
WARNING: no policy specified for oryx/127.13.75.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.13.75.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.13.75.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175691491353-13614-0/krb5kdc/oryx_127.13.75.130.keytab.
Entry for principal oryx/127.13.75.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175691491353-13614-0/krb5kdc/oryx_127.13.75.130.keytab.
I20260923 15:02:07.087347 13614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKC4soy/build/release/bin/kudu
/tmp/dist-test-taskKC4soy/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175691491353-13614-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175691491353-13614-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175691491353-13614-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175691491353-13614-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.75.130:0
--local_ip_for_outbound_sockets=127.13.75.130
--webserver_interface=127.13.75.130
--webserver_port=0
--tserver_master_addrs=127.13.75.190:40515
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175691491353-13614-0/krb5kdc/oryx_127.13.75.130.keytab
--principal=oryx/127.13.75.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.75.148:33291
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175691491353-13614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175691491353-13614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175691491353-13614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:02:07.176293 21338 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:02:07.176545 21338 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:02:07.176573 21338 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:02:07.177853 21338 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:02:07.177908 21338 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:02:07.177932 21338 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260923 15:02:07.177951 21338 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.75.130
I20260923 15:02:07.179625 21338 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.75.148:33291
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175691491353-13614-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175691491353-13614-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175691491353-13614-0/krb5kdc/oryx_127.13.75.130.keytab
--principal=oryx/127.13.75.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.75.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175691491353-13614-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.75.130
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.13.75.190:40515
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21338
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.75.130
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175691491353-13614-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:02:07.179909 21338 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:02:07.180183 21338 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:02:07.185815 21346 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:02:07.185827 21344 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:02:07.186167 21343 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:02:07.186218 21338 server_base.cc:1034] running on GCE node
I20260923 15:02:07.186493 21338 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:02:07.186770 21338 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:02:07.187959 21338 hybrid_clock.cc:648] HybridClock initialized: now 1790175727187955 us; error 45 us; skew 500 ppm
Sep 23 15:02:07 dist-test-slave-cnb0 krb5kdc[21081](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175727, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.75.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:02:07.190930 21338 init.cc:377] Logged in from keytab as oryx/127.13.75.130@KRBTEST.COM (short username oryx)
I20260923 15:02:07.191534 21338 webserver.cc:460] Webserver started at http://127.13.75.130:40593/ using document root <none> and password file <none>
I20260923 15:02:07.191798 21338 fs_manager.cc:359] Metadata directory not provided
I20260923 15:02:07.191879 21338 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:02:07.192050 21338 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:02:07.193251 21338 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175691491353-13614-0/minicluster-data/ts-1/data/instance:
uuid: "21a714af281940b497ab06ba85aaa27d"
format_stamp: "Formatted at 2026-09-23 15:02:07 on dist-test-slave-cnb0"
server_key: "15c80c39b51b12eb4e24a71af947ae84"
server_key_iv: "96afa60910160f66c2ca1b21b70c1a1e"
server_key_version: "encryptionkey@0"
I20260923 15:02:07.193681 21338 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175691491353-13614-0/minicluster-data/ts-1/wal/instance:
uuid: "21a714af281940b497ab06ba85aaa27d"
format_stamp: "Formatted at 2026-09-23 15:02:07 on dist-test-slave-cnb0"
server_key: "15c80c39b51b12eb4e24a71af947ae84"
server_key_iv: "96afa60910160f66c2ca1b21b70c1a1e"
server_key_version: "encryptionkey@0"
I20260923 15:02:07.195449 21338 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.004s
I20260923 15:02:07.196462 21353 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:02:07.196808 21338 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:02:07.196951 21338 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175691491353-13614-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175691491353-13614-0/minicluster-data/ts-1/wal
uuid: "21a714af281940b497ab06ba85aaa27d"
format_stamp: "Formatted at 2026-09-23 15:02:07 on dist-test-slave-cnb0"
server_key: "15c80c39b51b12eb4e24a71af947ae84"
server_key_iv: "96afa60910160f66c2ca1b21b70c1a1e"
server_key_version: "encryptionkey@0"
I20260923 15:02:07.197062 21338 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175691491353-13614-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175691491353-13614-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175691491353-13614-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:02:07.205483 21338 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:02:07.208293 21338 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:02:07.208483 21338 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:02:07.209139 21338 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 15:02:07.209192 21338 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 15:02:07.209230 21338 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 15:02:07.209257 21338 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 23 15:02:07 dist-test-slave-cnb0 krb5kdc[21081](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175727, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.75.130@KRBTEST.COM for oryx/127.13.75.190@KRBTEST.COM
I20260923 15:02:07.217353 21338 rpc_server.cc:307] RPC server started. Bound to: 127.13.75.130:46875
I20260923 15:02:07.217433 21468 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.75.130:46875 every 8 connection(s)
I20260923 15:02:07.217828 21338 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175691491353-13614-0/minicluster-data/ts-1/data/info.pb
I20260923 15:02:07.220444 21175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:07.209579 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.130:34493 (local address 127.13.75.190:40515)
0923 15:02:07.209650 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0923 15:02:07.209652 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:02:07.209984 (+ 332us) server_negotiation.cc:407] Connection header received
0923 15:02:07.210738 (+ 754us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:02:07.210739 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:02:07.210751 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:02:07.210797 (+ 46us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:02:07.211812 (+ 1015us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:07.212328 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:07.213552 (+ 1224us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:07.213774 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:07.213862 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:07.217101 (+ 3239us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:02:07.217103 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:02:07.217104 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:02:07.217114 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:02:07.218773 (+ 1659us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:02:07.219362 (+ 589us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:02:07.219364 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:02:07.219364 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:02:07.219446 (+ 82us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:02:07.219719 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:02:07.219721 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:02:07.219721 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:02:07.219857 (+ 136us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:02:07.219950 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0923 15:02:07.220257 (+ 307us) server_negotiation.cc:299] Negotiation successful
0923 15:02:07.220348 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260923 15:02:07.220642 21362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:07.209618 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:40515 (local address 127.13.75.130:34493)
0923 15:02:07.209826 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0923 15:02:07.209883 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0923 15:02:07.210616 (+ 733us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:02:07.210912 (+ 296us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:02:07.210917 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:02:07.211371 (+ 454us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:02:07.211691 (+ 320us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:07.211699 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:07.212524 (+ 825us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:07.212526 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:07.213421 (+ 895us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:07.213425 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:07.213875 (+ 450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:07.213876 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:07.214017 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:07.214714 (+ 697us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:02:07.214723 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:02:07.216968 (+ 2245us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:02:07.218876 (+ 1908us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:02:07.218955 (+ 79us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:02:07.218957 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:02:07.219244 (+ 287us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:02:07.219521 (+ 277us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:02:07.219522 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:02:07.219522 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:02:07.219639 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:02:07.219944 (+ 305us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:02:07.219951 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:02:07.220104 (+ 153us) client_negotiation.cc:769] Sending connection context
0923 15:02:07.220219 (+ 115us) client_negotiation.cc:240] Negotiation successful
0923 15:02:07.220410 (+ 191us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":97,"threads_started":1}
I20260923 15:02:07.221180 21469 heartbeater.cc:347] Connected to a master server at 127.13.75.190:40515
I20260923 15:02:07.221293 21469 heartbeater.cc:464] Registering TS with master...
I20260923 15:02:07.221506 21469 heartbeater.cc:511] Master 127.13.75.190:40515 requested a full tablet report, sending...
I20260923 15:02:07.221941 21125 ts_manager.cc:194] Registered new tserver with Master: 21a714af281940b497ab06ba85aaa27d (127.13.75.130:46875)
I20260923 15:02:07.222502 21125 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.13.75.130@KRBTEST.COM'} at 127.13.75.130:34493
I20260923 15:02:07.222788 13614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKC4soy/build/release/bin/kudu as pid 21338
I20260923 15:02:07.222867 13614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175691491353-13614-0/minicluster-data/ts-1/wal/instance
I20260923 15:02:07.223107 13614 external_mini_cluster.cc:1428] Setting key 3fe226139f3138c1640e8d30d36d84ae
I20260923 15:02:07.228581 21475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:07.222268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:40515 (local address 127.13.75.130:57207)
0923 15:02:07.222404 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0923 15:02:07.222415 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 15:02:07.222478 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:02:07.222620 (+ 142us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:02:07.222621 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:02:07.222828 (+ 207us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:02:07.222929 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:07.222932 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:07.223843 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:07.223844 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:07.224397 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:07.224399 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:07.224789 (+ 390us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:07.224790 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:07.224864 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:07.225375 (+ 511us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:02:07.225380 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:02:07.225692 (+ 312us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:02:07.227424 (+ 1732us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:02:07.227425 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:02:07.227426 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:02:07.227608 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:02:07.227838 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:02:07.227838 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:02:07.227839 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:02:07.227905 (+ 66us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:02:07.228207 (+ 302us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:02:07.228208 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:02:07.228259 (+ 51us) client_negotiation.cc:769] Sending connection context
0923 15:02:07.228348 (+ 89us) client_negotiation.cc:240] Negotiation successful
0923 15:02:07.228467 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":85,"threads_started":1}
I20260923 15:02:07.228572 21175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:07.222349 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.130:57207 (local address 127.13.75.190:40515)
0923 15:02:07.222392 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0923 15:02:07.222394 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:02:07.222469 (+ 75us) server_negotiation.cc:407] Connection header received
0923 15:02:07.222534 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:02:07.222535 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:02:07.222544 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:02:07.222615 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:02:07.223011 (+ 396us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:07.223742 (+ 731us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:07.224485 (+ 743us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:07.224698 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:07.224785 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:07.225803 (+ 1018us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:02:07.225804 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:02:07.225805 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:02:07.225812 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:02:07.227313 (+ 1501us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:02:07.227698 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:02:07.227699 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:02:07.227699 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:02:07.227761 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:02:07.227969 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:02:07.227970 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:02:07.227970 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:02:07.228133 (+ 163us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:02:07.228216 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0923 15:02:07.228373 (+ 157us) server_negotiation.cc:299] Negotiation successful
0923 15:02:07.228466 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":113,"server-negotiator.queue_time_us":23}
WARNING: no policy specified for oryx/127.13.75.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.13.75.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.13.75.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175691491353-13614-0/krb5kdc/oryx_127.13.75.131.keytab.
Entry for principal oryx/127.13.75.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175691491353-13614-0/krb5kdc/oryx_127.13.75.131.keytab.
I20260923 15:02:07.254962 13614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKC4soy/build/release/bin/kudu
/tmp/dist-test-taskKC4soy/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175691491353-13614-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175691491353-13614-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175691491353-13614-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175691491353-13614-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.75.131:0
--local_ip_for_outbound_sockets=127.13.75.131
--webserver_interface=127.13.75.131
--webserver_port=0
--tserver_master_addrs=127.13.75.190:40515
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175691491353-13614-0/krb5kdc/oryx_127.13.75.131.keytab
--principal=oryx/127.13.75.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.75.148:33291
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--enable_txn_system_client_init=true with env {KRB5CCNAME=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175691491353-13614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175691491353-13614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175691491353-13614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:02:07.343226 21478 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:02:07.343451 21478 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:02:07.343477 21478 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:02:07.344749 21478 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:02:07.344800 21478 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:02:07.344823 21478 flags.cc:425] Enabled experimental flag: --enable_txn_system_client_init=true
W20260923 15:02:07.344844 21478 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.75.131
I20260923 15:02:07.346474 21478 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.75.148:33291
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175691491353-13614-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175691491353-13614-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175691491353-13614-0/krb5kdc/oryx_127.13.75.131.keytab
--principal=oryx/127.13.75.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.75.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175691491353-13614-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.75.131
--webserver_port=0
--enable_txn_system_client_init=true
--tserver_master_addrs=127.13.75.190:40515
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21478
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.75.131
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175691491353-13614-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:02:07.346752 21478 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:02:07.347064 21478 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:02:07.353153 21484 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:02:07.353153 21483 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:02:07.353214 21478 server_base.cc:1034] running on GCE node
W20260923 15:02:07.353153 21486 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:02:07.353686 21478 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:02:07.353969 21478 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:02:07.355165 21478 hybrid_clock.cc:648] HybridClock initialized: now 1790175727355144 us; error 37 us; skew 500 ppm
Sep 23 15:02:07 dist-test-slave-cnb0 krb5kdc[21081](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175727, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.75.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:02:07.358028 21478 init.cc:377] Logged in from keytab as oryx/127.13.75.131@KRBTEST.COM (short username oryx)
I20260923 15:02:07.358566 21478 webserver.cc:460] Webserver started at http://127.13.75.131:45705/ using document root <none> and password file <none>
I20260923 15:02:07.358827 21478 fs_manager.cc:359] Metadata directory not provided
I20260923 15:02:07.358919 21478 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:02:07.359124 21478 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:02:07.360369 21478 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175691491353-13614-0/minicluster-data/ts-2/data/instance:
uuid: "b3dff0a518044ddeb2e65d3fbfb0ed68"
format_stamp: "Formatted at 2026-09-23 15:02:07 on dist-test-slave-cnb0"
server_key: "6b925ebde387797e9900d7aec1a06710"
server_key_iv: "fd26de900a0cf297e9f63204577abe37"
server_key_version: "encryptionkey@0"
I20260923 15:02:07.360759 21478 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175691491353-13614-0/minicluster-data/ts-2/wal/instance:
uuid: "b3dff0a518044ddeb2e65d3fbfb0ed68"
format_stamp: "Formatted at 2026-09-23 15:02:07 on dist-test-slave-cnb0"
server_key: "6b925ebde387797e9900d7aec1a06710"
server_key_iv: "fd26de900a0cf297e9f63204577abe37"
server_key_version: "encryptionkey@0"
I20260923 15:02:07.362188 21478 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260923 15:02:07.363148 21493 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:02:07.363348 21478 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260923 15:02:07.363508 21478 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175691491353-13614-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175691491353-13614-0/minicluster-data/ts-2/wal
uuid: "b3dff0a518044ddeb2e65d3fbfb0ed68"
format_stamp: "Formatted at 2026-09-23 15:02:07 on dist-test-slave-cnb0"
server_key: "6b925ebde387797e9900d7aec1a06710"
server_key_iv: "fd26de900a0cf297e9f63204577abe37"
server_key_version: "encryptionkey@0"
I20260923 15:02:07.363595 21478 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175691491353-13614-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175691491353-13614-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175691491353-13614-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:02:07.375254 21478 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:02:07.377727 21478 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:02:07.377923 21478 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:02:07.378468 21478 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 15:02:07.378540 21478 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 15:02:07.378604 21478 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 15:02:07.378635 21478 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
Sep 23 15:02:07 dist-test-slave-cnb0 krb5kdc[21081](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175727, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.75.131@KRBTEST.COM for oryx/127.13.75.190@KRBTEST.COM
I20260923 15:02:07.385775 21478 rpc_server.cc:307] RPC server started. Bound to: 127.13.75.131:39525
I20260923 15:02:07.386212 21478 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175691491353-13614-0/minicluster-data/ts-2/data/info.pb
I20260923 15:02:07.386654 21608 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.75.131:39525 every 8 connection(s)
I20260923 15:02:07.389446 21175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:07.379113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.131:60819 (local address 127.13.75.190:40515)
0923 15:02:07.379179 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0923 15:02:07.379182 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:02:07.379419 (+ 237us) server_negotiation.cc:407] Connection header received
0923 15:02:07.379986 (+ 567us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:02:07.379987 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:02:07.380000 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:02:07.380078 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:02:07.381040 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:07.381901 (+ 861us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:07.383366 (+ 1465us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:07.383721 (+ 355us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:07.383811 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:07.386330 (+ 2519us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:02:07.386332 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:02:07.386333 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:02:07.386342 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:02:07.387890 (+ 1548us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:02:07.388379 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:02:07.388380 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:02:07.388380 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:02:07.388457 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:02:07.388730 (+ 273us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:02:07.388731 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:02:07.388731 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:02:07.388915 (+ 184us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:02:07.388991 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0923 15:02:07.389267 (+ 276us) server_negotiation.cc:299] Negotiation successful
0923 15:02:07.389360 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260923 15:02:07.389524 21502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:07.379112 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:40515 (local address 127.13.75.131:60819)
0923 15:02:07.379277 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0923 15:02:07.379329 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0923 15:02:07.379868 (+ 539us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:02:07.380160 (+ 292us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:02:07.380166 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:02:07.380591 (+ 425us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:02:07.380910 (+ 319us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:07.380919 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:07.382010 (+ 1091us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:07.382012 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:07.383286 (+ 1274us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:07.383290 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:07.383807 (+ 517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:07.383808 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:07.383909 (+ 101us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:07.384534 (+ 625us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:02:07.384544 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:02:07.386203 (+ 1659us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:02:07.388044 (+ 1841us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:02:07.388051 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:02:07.388053 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:02:07.388281 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:02:07.388528 (+ 247us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:02:07.388529 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:02:07.388529 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:02:07.388640 (+ 111us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:02:07.388985 (+ 345us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:02:07.388989 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:02:07.389139 (+ 150us) client_negotiation.cc:769] Sending connection context
0923 15:02:07.389225 (+ 86us) client_negotiation.cc:240] Negotiation successful
0923 15:02:07.389357 (+ 132us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":91,"threads_started":1}
I20260923 15:02:07.390141 21609 heartbeater.cc:347] Connected to a master server at 127.13.75.190:40515
I20260923 15:02:07.390249 21609 heartbeater.cc:464] Registering TS with master...
I20260923 15:02:07.390434 21609 heartbeater.cc:511] Master 127.13.75.190:40515 requested a full tablet report, sending...
I20260923 15:02:07.390558 13614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKC4soy/build/release/bin/kudu as pid 21478
I20260923 15:02:07.390671 13614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175691491353-13614-0/minicluster-data/ts-2/wal/instance
I20260923 15:02:07.390859 21125 ts_manager.cc:194] Registered new tserver with Master: b3dff0a518044ddeb2e65d3fbfb0ed68 (127.13.75.131:39525)
I20260923 15:02:07.390957 13614 external_mini_cluster.cc:1428] Setting key 41b87497c9ad5354b32afd84eb8a4d3a
I20260923 15:02:07.391430 21125 master_service.cc:496] Signed X509 certificate for tserver {username='oryx', principal='oryx/127.13.75.131@KRBTEST.COM'} at 127.13.75.131:60819
I20260923 15:02:07.391983 13614 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260923 15:02:07.397742 21615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:07.391436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:40515 (local address 127.13.75.131:58153)
0923 15:02:07.391713 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0923 15:02:07.391725 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 15:02:07.391784 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:02:07.391952 (+ 168us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:02:07.391954 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:02:07.392151 (+ 197us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:02:07.392222 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:07.392224 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:07.392951 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:07.392952 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:07.393480 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:07.393482 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:07.393896 (+ 414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:07.393897 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:07.393969 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:07.394490 (+ 521us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:02:07.394495 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:02:07.394809 (+ 314us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:02:07.396594 (+ 1785us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:02:07.396595 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:02:07.396595 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:02:07.396776 (+ 181us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:02:07.397000 (+ 224us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:02:07.397000 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:02:07.397001 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:02:07.397068 (+ 67us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:02:07.397363 (+ 295us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:02:07.397364 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:02:07.397416 (+ 52us) client_negotiation.cc:769] Sending connection context
0923 15:02:07.397501 (+ 85us) client_negotiation.cc:240] Negotiation successful
0923 15:02:07.397628 (+ 127us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":249,"thread_start_us":74,"threads_started":1}
I20260923 15:02:07.397730 21175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:07.391481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.131:58153 (local address 127.13.75.190:40515)
0923 15:02:07.391526 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0923 15:02:07.391528 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:02:07.391814 (+ 286us) server_negotiation.cc:407] Connection header received
0923 15:02:07.391841 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:02:07.391842 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:02:07.391854 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:02:07.391932 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:02:07.392332 (+ 400us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:07.392847 (+ 515us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:07.393588 (+ 741us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:07.393800 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:07.393894 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:07.395040 (+ 1146us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:02:07.395042 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:02:07.395042 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:02:07.395050 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:02:07.396451 (+ 1401us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:02:07.396863 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:02:07.396865 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:02:07.396865 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:02:07.396925 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:02:07.397149 (+ 224us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:02:07.397150 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:02:07.397150 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:02:07.397272 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:02:07.397370 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0923 15:02:07.397540 (+ 170us) server_negotiation.cc:299] Negotiation successful
0923 15:02:07.397626 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
Sep 23 15:02:07 dist-test-slave-cnb0 krb5kdc[21081](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175727, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-user@KRBTEST.COM:
Sep 23 15:02:07 dist-test-slave-cnb0 krb5kdc[21081](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.13.75.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 15:02:07 dist-test-slave-cnb0 krb5kdc[21081](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-user@KRBTEST.COM for kudu/127.13.75.190@KRBTEST.COM, Server not found in Kerberos database
W20260923 15:02:07.417898 21175 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 15:02:07.410802 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35122 (local address 127.13.75.190:40515)
0923 15:02:07.411024 (+ 222us) server_negotiation.cc:206] Beginning negotiation
0923 15:02:07.411027 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:02:07.411140 (+ 113us) server_negotiation.cc:407] Connection header received
0923 15:02:07.411169 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:02:07.411170 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:02:07.411181 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:02:07.411278 (+ 97us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:02:07.411914 (+ 636us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:07.412560 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:07.413372 (+ 812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:07.413567 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:07.413652 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:07.417798 (+ 4146us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35122: BlockingRecv error: recv got EOF from 127.0.0.1:35122 (error 108)
Metrics: {"server-negotiator.queue_time_us":187}
Sep 23 15:02:07 dist-test-slave-cnb0 krb5kdc[21081](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175727, etypes {rep=17 tkt=17 ses=17}, test-user@KRBTEST.COM for oryx/127.13.75.190@KRBTEST.COM
I20260923 15:02:07.427816 21175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:07.418863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35124 (local address 127.13.75.190:40515)
0923 15:02:07.418953 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0923 15:02:07.418957 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 15:02:07.419015 (+ 58us) server_negotiation.cc:407] Connection header received
0923 15:02:07.419080 (+ 65us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:02:07.419081 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:02:07.419090 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:02:07.419152 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:02:07.419729 (+ 577us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:07.420403 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:07.421416 (+ 1013us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:07.421656 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:07.421737 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:07.424434 (+ 2697us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:02:07.424436 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:02:07.424436 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:02:07.424446 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:02:07.426243 (+ 1797us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:02:07.426703 (+ 460us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:02:07.426705 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:02:07.426705 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:02:07.426768 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:02:07.427136 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:02:07.427137 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:02:07.427137 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:02:07.427271 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:02:07.427517 (+ 246us) server_negotiation.cc:1035] Waiting for connection context
0923 15:02:07.427615 (+ 98us) server_negotiation.cc:299] Negotiation successful
0923 15:02:07.427694 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":67}
I20260923 15:02:07.430024 21125 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35124:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 15:02:07.431854 21125 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 15:02:07.439711 21635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:07.435013 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52954 (local address 127.13.75.130:46875)
0923 15:02:07.435358 (+ 345us) server_negotiation.cc:206] Beginning negotiation
0923 15:02:07.435363 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 15:02:07.435580 (+ 217us) server_negotiation.cc:407] Connection header received
0923 15:02:07.435636 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:02:07.435638 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:02:07.435655 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:02:07.435725 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:02:07.436020 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:07.436917 (+ 897us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:07.438503 (+ 1586us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:07.439140 (+ 637us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:07.439245 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:07.439330 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0923 15:02:07.439503 (+ 173us) server_negotiation.cc:299] Negotiation successful
0923 15:02:07.439548 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":256,"thread_start_us":110,"threads_started":1}
I20260923 15:02:07.439819 21633 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:07.434711 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.131:39525 (local address 127.0.0.1:56734)
0923 15:02:07.435805 (+ 1094us) negotiation.cc:107] Waiting for socket to connect
0923 15:02:07.435815 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 15:02:07.435869 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:02:07.436086 (+ 217us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:02:07.436087 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:02:07.436090 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:02:07.436133 (+ 43us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:07.436134 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:07.437484 (+ 1350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:07.437485 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:07.438270 (+ 785us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:07.438273 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:07.439408 (+ 1135us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:07.439408 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:07.439636 (+ 228us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:07.439640 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 15:02:07.439688 (+ 48us) client_negotiation.cc:240] Negotiation successful
0923 15:02:07.439729 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1074,"thread_start_us":111,"threads_started":1}
I20260923 15:02:07.440064 21632 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:07.434695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.130:46875 (local address 127.0.0.1:52954)
0923 15:02:07.435423 (+ 728us) negotiation.cc:107] Waiting for socket to connect
0923 15:02:07.435435 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 15:02:07.435490 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:02:07.435749 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:02:07.435750 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:02:07.435751 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:02:07.435872 (+ 121us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:07.435874 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:07.437008 (+ 1134us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:07.437009 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:07.438393 (+ 1384us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:07.438396 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:07.439249 (+ 853us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:07.439250 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:07.439350 (+ 100us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:07.439365 (+ 15us) client_negotiation.cc:769] Sending connection context
0923 15:02:07.439482 (+ 117us) client_negotiation.cc:240] Negotiation successful
0923 15:02:07.439984 (+ 502us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":694,"mutex_wait_us":446,"thread_start_us":68,"threads_started":1}
I20260923 15:02:07.440168 21631 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:07.434691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.129:38767 (local address 127.0.0.1:35798)
0923 15:02:07.434871 (+ 180us) negotiation.cc:107] Waiting for socket to connect
0923 15:02:07.434955 (+ 84us) client_negotiation.cc:174] Beginning negotiation
0923 15:02:07.435026 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:02:07.435741 (+ 715us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:02:07.435746 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:02:07.435749 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:02:07.435876 (+ 127us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:07.435877 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:07.437238 (+ 1361us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:07.437239 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:07.438393 (+ 1154us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:07.438396 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:07.439343 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:07.439345 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:07.439498 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:07.439513 (+ 15us) client_negotiation.cc:769] Sending connection context
0923 15:02:07.439567 (+ 54us) client_negotiation.cc:240] Negotiation successful
0923 15:02:07.440097 (+ 530us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"mutex_wait_us":512,"thread_start_us":95,"threads_started":1}
I20260923 15:02:07.440183 21634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:07.434927 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56734 (local address 127.13.75.131:39525)
0923 15:02:07.435579 (+ 652us) server_negotiation.cc:206] Beginning negotiation
0923 15:02:07.435583 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 15:02:07.435928 (+ 345us) server_negotiation.cc:407] Connection header received
0923 15:02:07.435983 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:02:07.435985 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:02:07.436003 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:02:07.436079 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:02:07.436191 (+ 112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:07.437385 (+ 1194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:07.438382 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:07.439187 (+ 805us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:07.439288 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:07.439378 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0923 15:02:07.440037 (+ 659us) server_negotiation.cc:299] Negotiation successful
0923 15:02:07.440091 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":616,"thread_start_us":168,"threads_started":1}
I20260923 15:02:07.440848 21403 tablet_service.cc:1467] Processing CreateTablet for tablet bd97489579bf4605b3737c2fc0bc5e01 (DEFAULT_TABLE table=test-table [id=defdea7049b243bf919fbbfa4bd0b265]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 15:02:07.440848 21543 tablet_service.cc:1467] Processing CreateTablet for tablet bd97489579bf4605b3737c2fc0bc5e01 (DEFAULT_TABLE table=test-table [id=defdea7049b243bf919fbbfa4bd0b265]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 15:02:07.441267 21403 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bd97489579bf4605b3737c2fc0bc5e01. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:02:07.441267 21543 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bd97489579bf4605b3737c2fc0bc5e01. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:02:07.445659 21636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:07.435250 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35798 (local address 127.13.75.129:38767)
0923 15:02:07.435543 (+ 293us) server_negotiation.cc:206] Beginning negotiation
0923 15:02:07.435547 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 15:02:07.435557 (+ 10us) server_negotiation.cc:407] Connection header received
0923 15:02:07.435600 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:02:07.435603 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:02:07.435619 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:02:07.435707 (+ 88us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:02:07.435978 (+ 271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:07.437148 (+ 1170us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:07.438497 (+ 1349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:07.439246 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:07.439650 (+ 404us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:07.439751 (+ 101us) server_negotiation.cc:1035] Waiting for connection context
0923 15:02:07.439780 (+ 29us) server_negotiation.cc:299] Negotiation successful
0923 15:02:07.440358 (+ 578us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":190,"thread_start_us":123,"threads_started":1}
I20260923 15:02:07.445614 21638 tablet_bootstrap.cc:492] T bd97489579bf4605b3737c2fc0bc5e01 P 21a714af281940b497ab06ba85aaa27d: Bootstrap starting.
I20260923 15:02:07.446153 21637 tablet_bootstrap.cc:492] T bd97489579bf4605b3737c2fc0bc5e01 P b3dff0a518044ddeb2e65d3fbfb0ed68: Bootstrap starting.
I20260923 15:02:07.446579 21638 tablet_bootstrap.cc:654] T bd97489579bf4605b3737c2fc0bc5e01 P 21a714af281940b497ab06ba85aaa27d: Neither blocks nor log segments found. Creating new log.
I20260923 15:02:07.446542 21263 tablet_service.cc:1467] Processing CreateTablet for tablet bd97489579bf4605b3737c2fc0bc5e01 (DEFAULT_TABLE table=test-table [id=defdea7049b243bf919fbbfa4bd0b265]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 15:02:07.447000 21638 log.cc:824] T bd97489579bf4605b3737c2fc0bc5e01 P 21a714af281940b497ab06ba85aaa27d: Log is configured to *not* fsync() on all Append() calls
I20260923 15:02:07.447028 21637 tablet_bootstrap.cc:654] T bd97489579bf4605b3737c2fc0bc5e01 P b3dff0a518044ddeb2e65d3fbfb0ed68: Neither blocks nor log segments found. Creating new log.
I20260923 15:02:07.447045 21263 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet bd97489579bf4605b3737c2fc0bc5e01. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:02:07.447468 21637 log.cc:824] T bd97489579bf4605b3737c2fc0bc5e01 P b3dff0a518044ddeb2e65d3fbfb0ed68: Log is configured to *not* fsync() on all Append() calls
I20260923 15:02:07.448006 21638 tablet_bootstrap.cc:492] T bd97489579bf4605b3737c2fc0bc5e01 P 21a714af281940b497ab06ba85aaa27d: No bootstrap required, opened a new log
I20260923 15:02:07.448091 21638 ts_tablet_manager.cc:1397] T bd97489579bf4605b3737c2fc0bc5e01 P 21a714af281940b497ab06ba85aaa27d: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.001s
I20260923 15:02:07.448315 21637 tablet_bootstrap.cc:492] T bd97489579bf4605b3737c2fc0bc5e01 P b3dff0a518044ddeb2e65d3fbfb0ed68: No bootstrap required, opened a new log
I20260923 15:02:07.448385 21637 ts_tablet_manager.cc:1397] T bd97489579bf4605b3737c2fc0bc5e01 P b3dff0a518044ddeb2e65d3fbfb0ed68: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260923 15:02:07.450340 21638 raft_consensus.cc:348] T bd97489579bf4605b3737c2fc0bc5e01 P 21a714af281940b497ab06ba85aaa27d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2f18bf473b04a23a122a756e8f646a3" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 38767 } } peers { permanent_uuid: "21a714af281940b497ab06ba85aaa27d" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 46875 } } peers { permanent_uuid: "b3dff0a518044ddeb2e65d3fbfb0ed68" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 39525 } }
I20260923 15:02:07.450570 21638 raft_consensus.cc:374] T bd97489579bf4605b3737c2fc0bc5e01 P 21a714af281940b497ab06ba85aaa27d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:02:07.450495 21637 raft_consensus.cc:348] T bd97489579bf4605b3737c2fc0bc5e01 P b3dff0a518044ddeb2e65d3fbfb0ed68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2f18bf473b04a23a122a756e8f646a3" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 38767 } } peers { permanent_uuid: "21a714af281940b497ab06ba85aaa27d" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 46875 } } peers { permanent_uuid: "b3dff0a518044ddeb2e65d3fbfb0ed68" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 39525 } }
I20260923 15:02:07.450609 21638 raft_consensus.cc:729] T bd97489579bf4605b3737c2fc0bc5e01 P 21a714af281940b497ab06ba85aaa27d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 21a714af281940b497ab06ba85aaa27d, State: Initialized, Role: FOLLOWER
I20260923 15:02:07.450626 21637 raft_consensus.cc:374] T bd97489579bf4605b3737c2fc0bc5e01 P b3dff0a518044ddeb2e65d3fbfb0ed68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:02:07.450656 21637 raft_consensus.cc:729] T bd97489579bf4605b3737c2fc0bc5e01 P b3dff0a518044ddeb2e65d3fbfb0ed68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3dff0a518044ddeb2e65d3fbfb0ed68, State: Initialized, Role: FOLLOWER
I20260923 15:02:07.450755 21637 consensus_queue.cc:260] T bd97489579bf4605b3737c2fc0bc5e01 P b3dff0a518044ddeb2e65d3fbfb0ed68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2f18bf473b04a23a122a756e8f646a3" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 38767 } } peers { permanent_uuid: "21a714af281940b497ab06ba85aaa27d" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 46875 } } peers { permanent_uuid: "b3dff0a518044ddeb2e65d3fbfb0ed68" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 39525 } }
I20260923 15:02:07.450783 21638 consensus_queue.cc:260] T bd97489579bf4605b3737c2fc0bc5e01 P 21a714af281940b497ab06ba85aaa27d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2f18bf473b04a23a122a756e8f646a3" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 38767 } } peers { permanent_uuid: "21a714af281940b497ab06ba85aaa27d" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 46875 } } peers { permanent_uuid: "b3dff0a518044ddeb2e65d3fbfb0ed68" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 39525 } }
I20260923 15:02:07.451032 21641 tablet_bootstrap.cc:492] T bd97489579bf4605b3737c2fc0bc5e01 P f2f18bf473b04a23a122a756e8f646a3: Bootstrap starting.
I20260923 15:02:07.451102 21637 ts_tablet_manager.cc:1428] T bd97489579bf4605b3737c2fc0bc5e01 P b3dff0a518044ddeb2e65d3fbfb0ed68: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260923 15:02:07.451102 21638 ts_tablet_manager.cc:1428] T bd97489579bf4605b3737c2fc0bc5e01 P 21a714af281940b497ab06ba85aaa27d: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260923 15:02:07.451223 21469 heartbeater.cc:503] Master 127.13.75.190:40515 was elected leader, sending a full tablet report...
I20260923 15:02:07.451323 21609 heartbeater.cc:503] Master 127.13.75.190:40515 was elected leader, sending a full tablet report...
I20260923 15:02:07.451834 21641 tablet_bootstrap.cc:654] T bd97489579bf4605b3737c2fc0bc5e01 P f2f18bf473b04a23a122a756e8f646a3: Neither blocks nor log segments found. Creating new log.
I20260923 15:02:07.452299 21641 log.cc:824] T bd97489579bf4605b3737c2fc0bc5e01 P f2f18bf473b04a23a122a756e8f646a3: Log is configured to *not* fsync() on all Append() calls
I20260923 15:02:07.453250 21641 tablet_bootstrap.cc:492] T bd97489579bf4605b3737c2fc0bc5e01 P f2f18bf473b04a23a122a756e8f646a3: No bootstrap required, opened a new log
I20260923 15:02:07.453310 21641 ts_tablet_manager.cc:1397] T bd97489579bf4605b3737c2fc0bc5e01 P f2f18bf473b04a23a122a756e8f646a3: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260923 15:02:07.454811 21641 raft_consensus.cc:348] T bd97489579bf4605b3737c2fc0bc5e01 P f2f18bf473b04a23a122a756e8f646a3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2f18bf473b04a23a122a756e8f646a3" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 38767 } } peers { permanent_uuid: "21a714af281940b497ab06ba85aaa27d" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 46875 } } peers { permanent_uuid: "b3dff0a518044ddeb2e65d3fbfb0ed68" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 39525 } }
I20260923 15:02:07.455042 21641 raft_consensus.cc:374] T bd97489579bf4605b3737c2fc0bc5e01 P f2f18bf473b04a23a122a756e8f646a3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:02:07.455089 21641 raft_consensus.cc:729] T bd97489579bf4605b3737c2fc0bc5e01 P f2f18bf473b04a23a122a756e8f646a3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2f18bf473b04a23a122a756e8f646a3, State: Initialized, Role: FOLLOWER
I20260923 15:02:07.455229 21641 consensus_queue.cc:260] T bd97489579bf4605b3737c2fc0bc5e01 P f2f18bf473b04a23a122a756e8f646a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2f18bf473b04a23a122a756e8f646a3" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 38767 } } peers { permanent_uuid: "21a714af281940b497ab06ba85aaa27d" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 46875 } } peers { permanent_uuid: "b3dff0a518044ddeb2e65d3fbfb0ed68" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 39525 } }
I20260923 15:02:07.455528 21641 ts_tablet_manager.cc:1428] T bd97489579bf4605b3737c2fc0bc5e01 P f2f18bf473b04a23a122a756e8f646a3: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260923 15:02:07.455655 21329 heartbeater.cc:503] Master 127.13.75.190:40515 was elected leader, sending a full tablet report...
W20260923 15:02:07.468713 21470 tablet.cc:2369] T bd97489579bf4605b3737c2fc0bc5e01 P 21a714af281940b497ab06ba85aaa27d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 15:02:07.525480 21643 raft_consensus.cc:482] T bd97489579bf4605b3737c2fc0bc5e01 P 21a714af281940b497ab06ba85aaa27d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 15:02:07.525612 21643 raft_consensus.cc:504] T bd97489579bf4605b3737c2fc0bc5e01 P 21a714af281940b497ab06ba85aaa27d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2f18bf473b04a23a122a756e8f646a3" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 38767 } } peers { permanent_uuid: "21a714af281940b497ab06ba85aaa27d" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 46875 } } peers { permanent_uuid: "b3dff0a518044ddeb2e65d3fbfb0ed68" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 39525 } }
I20260923 15:02:07.526064 21643 leader_election.cc:288] T bd97489579bf4605b3737c2fc0bc5e01 P 21a714af281940b497ab06ba85aaa27d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f2f18bf473b04a23a122a756e8f646a3 (127.13.75.129:38767), b3dff0a518044ddeb2e65d3fbfb0ed68 (127.13.75.131:39525)
I20260923 15:02:07.530454 21636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:07.526449 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.130:54073 (local address 127.13.75.129:38767)
0923 15:02:07.526497 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0923 15:02:07.526499 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:02:07.526511 (+ 12us) server_negotiation.cc:407] Connection header received
0923 15:02:07.526545 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:02:07.526546 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:02:07.526557 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:02:07.526637 (+ 80us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:02:07.526953 (+ 316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:07.527708 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:07.529357 (+ 1649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:07.529957 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:07.530037 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:07.530049 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0923 15:02:07.530343 (+ 294us) server_negotiation.cc:299] Negotiation successful
0923 15:02:07.530394 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260923 15:02:07.530531 21362 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:07.526332 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.129:38767 (local address 127.13.75.130:54073)
0923 15:02:07.526404 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0923 15:02:07.526417 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0923 15:02:07.526470 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:02:07.526722 (+ 252us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:02:07.526723 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:02:07.526726 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:02:07.526848 (+ 122us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:07.526851 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:07.527826 (+ 975us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:07.527827 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:07.529254 (+ 1427us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:07.529257 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:07.530064 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:07.530065 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:07.530196 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:07.530200 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 15:02:07.530278 (+ 78us) client_negotiation.cc:240] Negotiation successful
0923 15:02:07.530341 (+ 63us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":52,"mutex_wait_us":3}
I20260923 15:02:07.530669 21646 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:07.526331 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.131:39525 (local address 127.13.75.130:39951)
0923 15:02:07.526540 (+ 209us) negotiation.cc:107] Waiting for socket to connect
0923 15:02:07.526613 (+ 73us) client_negotiation.cc:174] Beginning negotiation
0923 15:02:07.526847 (+ 234us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:02:07.527079 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:02:07.527080 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:02:07.527082 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:02:07.527152 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:07.527153 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:07.528031 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:07.528032 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:07.529254 (+ 1222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:07.529257 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:07.530064 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:07.530065 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:07.530458 (+ 393us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:07.530463 (+ 5us) client_negotiation.cc:769] Sending connection context
0923 15:02:07.530524 (+ 61us) client_negotiation.cc:240] Negotiation successful
0923 15:02:07.530568 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":193,"mutex_wait_us":51,"thread_start_us":100,"threads_started":1}
I20260923 15:02:07.531032 21634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:07.526448 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.130:39951 (local address 127.13.75.131:39525)
0923 15:02:07.526546 (+ 98us) server_negotiation.cc:206] Beginning negotiation
0923 15:02:07.526548 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:02:07.526844 (+ 296us) server_negotiation.cc:407] Connection header received
0923 15:02:07.526952 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:02:07.526953 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:02:07.526964 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:02:07.527029 (+ 65us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:02:07.527240 (+ 211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:07.527931 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:07.529357 (+ 1426us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:07.529932 (+ 575us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:07.530044 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:07.530052 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0923 15:02:07.530911 (+ 859us) server_negotiation.cc:299] Negotiation successful
0923 15:02:07.530963 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260923 15:02:07.530977 21283 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd97489579bf4605b3737c2fc0bc5e01" candidate_uuid: "21a714af281940b497ab06ba85aaa27d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2f18bf473b04a23a122a756e8f646a3" is_pre_election: true
I20260923 15:02:07.531167 21283 raft_consensus.cc:2454] T bd97489579bf4605b3737c2fc0bc5e01 P f2f18bf473b04a23a122a756e8f646a3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 21a714af281940b497ab06ba85aaa27d in term 0.
I20260923 15:02:07.531438 21356 leader_election.cc:302] T bd97489579bf4605b3737c2fc0bc5e01 P 21a714af281940b497ab06ba85aaa27d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 21a714af281940b497ab06ba85aaa27d, f2f18bf473b04a23a122a756e8f646a3; no voters:
I20260923 15:02:07.531474 21563 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd97489579bf4605b3737c2fc0bc5e01" candidate_uuid: "21a714af281940b497ab06ba85aaa27d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3dff0a518044ddeb2e65d3fbfb0ed68" is_pre_election: true
I20260923 15:02:07.531628 21563 raft_consensus.cc:2454] T bd97489579bf4605b3737c2fc0bc5e01 P b3dff0a518044ddeb2e65d3fbfb0ed68 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 21a714af281940b497ab06ba85aaa27d in term 0.
I20260923 15:02:07.531615 21643 raft_consensus.cc:2781] T bd97489579bf4605b3737c2fc0bc5e01 P 21a714af281940b497ab06ba85aaa27d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 15:02:07.531682 21643 raft_consensus.cc:482] T bd97489579bf4605b3737c2fc0bc5e01 P 21a714af281940b497ab06ba85aaa27d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 15:02:07.531713 21643 raft_consensus.cc:3037] T bd97489579bf4605b3737c2fc0bc5e01 P 21a714af281940b497ab06ba85aaa27d [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:02:07.532485 21643 raft_consensus.cc:504] T bd97489579bf4605b3737c2fc0bc5e01 P 21a714af281940b497ab06ba85aaa27d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2f18bf473b04a23a122a756e8f646a3" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 38767 } } peers { permanent_uuid: "21a714af281940b497ab06ba85aaa27d" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 46875 } } peers { permanent_uuid: "b3dff0a518044ddeb2e65d3fbfb0ed68" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 39525 } }
I20260923 15:02:07.532645 21643 leader_election.cc:288] T bd97489579bf4605b3737c2fc0bc5e01 P 21a714af281940b497ab06ba85aaa27d [CANDIDATE]: Term 1 election: Requested vote from peers f2f18bf473b04a23a122a756e8f646a3 (127.13.75.129:38767), b3dff0a518044ddeb2e65d3fbfb0ed68 (127.13.75.131:39525)
I20260923 15:02:07.532788 21563 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd97489579bf4605b3737c2fc0bc5e01" candidate_uuid: "21a714af281940b497ab06ba85aaa27d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3dff0a518044ddeb2e65d3fbfb0ed68"
I20260923 15:02:07.532891 21563 raft_consensus.cc:3037] T bd97489579bf4605b3737c2fc0bc5e01 P b3dff0a518044ddeb2e65d3fbfb0ed68 [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:02:07.532855 21283 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "bd97489579bf4605b3737c2fc0bc5e01" candidate_uuid: "21a714af281940b497ab06ba85aaa27d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2f18bf473b04a23a122a756e8f646a3"
I20260923 15:02:07.533003 21283 raft_consensus.cc:3037] T bd97489579bf4605b3737c2fc0bc5e01 P f2f18bf473b04a23a122a756e8f646a3 [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:02:07.533710 21563 raft_consensus.cc:2454] T bd97489579bf4605b3737c2fc0bc5e01 P b3dff0a518044ddeb2e65d3fbfb0ed68 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 21a714af281940b497ab06ba85aaa27d in term 1.
I20260923 15:02:07.533725 21283 raft_consensus.cc:2454] T bd97489579bf4605b3737c2fc0bc5e01 P f2f18bf473b04a23a122a756e8f646a3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 21a714af281940b497ab06ba85aaa27d in term 1.
I20260923 15:02:07.533967 21357 leader_election.cc:302] T bd97489579bf4605b3737c2fc0bc5e01 P 21a714af281940b497ab06ba85aaa27d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 21a714af281940b497ab06ba85aaa27d, b3dff0a518044ddeb2e65d3fbfb0ed68; no voters:
I20260923 15:02:07.534129 21643 raft_consensus.cc:2781] T bd97489579bf4605b3737c2fc0bc5e01 P 21a714af281940b497ab06ba85aaa27d [term 1 FOLLOWER]: Leader election won for term 1
I20260923 15:02:07.534291 21643 raft_consensus.cc:686] T bd97489579bf4605b3737c2fc0bc5e01 P 21a714af281940b497ab06ba85aaa27d [term 1 LEADER]: Becoming Leader. State: Replica: 21a714af281940b497ab06ba85aaa27d, State: Running, Role: LEADER
I20260923 15:02:07.534361 21643 consensus_queue.cc:237] T bd97489579bf4605b3737c2fc0bc5e01 P 21a714af281940b497ab06ba85aaa27d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2f18bf473b04a23a122a756e8f646a3" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 38767 } } peers { permanent_uuid: "21a714af281940b497ab06ba85aaa27d" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 46875 } } peers { permanent_uuid: "b3dff0a518044ddeb2e65d3fbfb0ed68" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 39525 } }
I20260923 15:02:07.535578 21126 catalog_manager.cc:5495] T bd97489579bf4605b3737c2fc0bc5e01 P 21a714af281940b497ab06ba85aaa27d reported cstate change: term changed from 0 to 1, leader changed from <none> to 21a714af281940b497ab06ba85aaa27d (127.13.75.130). New cstate: current_term: 1 leader_uuid: "21a714af281940b497ab06ba85aaa27d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2f18bf473b04a23a122a756e8f646a3" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 38767 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "21a714af281940b497ab06ba85aaa27d" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 46875 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b3dff0a518044ddeb2e65d3fbfb0ed68" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 39525 } health_report { overall_health: UNKNOWN } } }
W20260923 15:02:07.549577 21330 tablet.cc:2369] T bd97489579bf4605b3737c2fc0bc5e01 P f2f18bf473b04a23a122a756e8f646a3: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 15:02:07.557804 21635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:07.555056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52970 (local address 127.13.75.130:46875)
0923 15:02:07.555121 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0923 15:02:07.555123 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:02:07.555132 (+ 9us) server_negotiation.cc:407] Connection header received
0923 15:02:07.555178 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:02:07.555179 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:02:07.555196 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:02:07.555293 (+ 97us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 15:02:07.555448 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:07.555922 (+ 474us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:07.556919 (+ 997us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:07.557119 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:07.557207 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:07.557449 (+ 242us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 15:02:07.557562 (+ 113us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 15:02:07.557667 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0923 15:02:07.557685 (+ 18us) server_negotiation.cc:299] Negotiation successful
0923 15:02:07.557710 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260923 15:02:07.559834 21563 raft_consensus.cc:1261] T bd97489579bf4605b3737c2fc0bc5e01 P b3dff0a518044ddeb2e65d3fbfb0ed68 [term 1 FOLLOWER]: Refusing update from remote peer 21a714af281940b497ab06ba85aaa27d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 15:02:07.559821 21283 raft_consensus.cc:1261] T bd97489579bf4605b3737c2fc0bc5e01 P f2f18bf473b04a23a122a756e8f646a3 [term 1 FOLLOWER]: Refusing update from remote peer 21a714af281940b497ab06ba85aaa27d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 15:02:07.560277 21647 consensus_queue.cc:1035] T bd97489579bf4605b3737c2fc0bc5e01 P 21a714af281940b497ab06ba85aaa27d [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2f18bf473b04a23a122a756e8f646a3" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 38767 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 15:02:07.560436 21643 consensus_queue.cc:1035] T bd97489579bf4605b3737c2fc0bc5e01 P 21a714af281940b497ab06ba85aaa27d [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3dff0a518044ddeb2e65d3fbfb0ed68" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 39525 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 15:02:07.563175 21651 mvcc.cc:205] Tried to move back new op lower bound from 7332559780082077696 to 7332559779981565952. Current Snapshot: MvccSnapshot[applied={T|T < 7332559780082077696}]
I20260923 15:02:07.565136 21125 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35124:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 15:02:07.565243 21125 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35124:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 15:02:07.566110 21125 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 385ef6dbf6774aa09aff37a6dcf6afd0: Sending DeleteTablet for 3 replicas of tablet bd97489579bf4605b3737c2fc0bc5e01
I20260923 15:02:07.566466 21403 tablet_service.cc:1514] Processing DeleteTablet for tablet bd97489579bf4605b3737c2fc0bc5e01 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 15:02:07 UTC) from {username='oryx', principal='oryx/127.13.75.190@KRBTEST.COM'} at 127.0.0.1:52954
I20260923 15:02:07.566540 21263 tablet_service.cc:1514] Processing DeleteTablet for tablet bd97489579bf4605b3737c2fc0bc5e01 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 15:02:07 UTC) from {username='oryx', principal='oryx/127.13.75.190@KRBTEST.COM'} at 127.0.0.1:35798
I20260923 15:02:07.566582 21543 tablet_service.cc:1514] Processing DeleteTablet for tablet bd97489579bf4605b3737c2fc0bc5e01 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 15:02:07 UTC) from {username='oryx', principal='oryx/127.13.75.190@KRBTEST.COM'} at 127.0.0.1:56734
I20260923 15:02:07.566709 21125 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35124:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 15:02:07.566866 21125 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 15:02:07.566906 21661 tablet_replica.cc:330] T bd97489579bf4605b3737c2fc0bc5e01 P 21a714af281940b497ab06ba85aaa27d: stopping tablet replica
I20260923 15:02:07.567019 21661 raft_consensus.cc:2229] T bd97489579bf4605b3737c2fc0bc5e01 P 21a714af281940b497ab06ba85aaa27d [term 1 LEADER]: Raft consensus shutting down.
I20260923 15:02:07.567257 21661 raft_consensus.cc:2258] T bd97489579bf4605b3737c2fc0bc5e01 P 21a714af281940b497ab06ba85aaa27d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 15:02:07.567945 21661 ts_tablet_manager.cc:1905] T bd97489579bf4605b3737c2fc0bc5e01 P 21a714af281940b497ab06ba85aaa27d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 15:02:07.568169 21263 tablet_service.cc:1467] Processing CreateTablet for tablet 903409fdfec246f88f037d3420c1a942 (DEFAULT_TABLE table=test-table [id=c68093850b9b42aea4a70ae4a0f355c0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 15:02:07.568207 21543 tablet_service.cc:1467] Processing CreateTablet for tablet 903409fdfec246f88f037d3420c1a942 (DEFAULT_TABLE table=test-table [id=c68093850b9b42aea4a70ae4a0f355c0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 15:02:07.568293 21263 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 903409fdfec246f88f037d3420c1a942. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:02:07.568320 21543 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 903409fdfec246f88f037d3420c1a942. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:02:07.569723 21660 tablet_replica.cc:330] T bd97489579bf4605b3737c2fc0bc5e01 P f2f18bf473b04a23a122a756e8f646a3: stopping tablet replica
I20260923 15:02:07.569785 21641 tablet_bootstrap.cc:492] T 903409fdfec246f88f037d3420c1a942 P f2f18bf473b04a23a122a756e8f646a3: Bootstrap starting.
I20260923 15:02:07.569839 21660 raft_consensus.cc:2229] T bd97489579bf4605b3737c2fc0bc5e01 P f2f18bf473b04a23a122a756e8f646a3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 15:02:07.569947 21660 raft_consensus.cc:2258] T bd97489579bf4605b3737c2fc0bc5e01 P f2f18bf473b04a23a122a756e8f646a3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 15:02:07.570179 21641 tablet_bootstrap.cc:654] T 903409fdfec246f88f037d3420c1a942 P f2f18bf473b04a23a122a756e8f646a3: Neither blocks nor log segments found. Creating new log.
I20260923 15:02:07.570412 21662 tablet_replica.cc:330] T bd97489579bf4605b3737c2fc0bc5e01 P b3dff0a518044ddeb2e65d3fbfb0ed68: stopping tablet replica
I20260923 15:02:07.570523 21637 tablet_bootstrap.cc:492] T 903409fdfec246f88f037d3420c1a942 P b3dff0a518044ddeb2e65d3fbfb0ed68: Bootstrap starting.
I20260923 15:02:07.570663 21661 ts_tablet_manager.cc:1918] T bd97489579bf4605b3737c2fc0bc5e01 P 21a714af281940b497ab06ba85aaa27d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 15:02:07.570739 21661 log.cc:1196] T bd97489579bf4605b3737c2fc0bc5e01 P 21a714af281940b497ab06ba85aaa27d: Deleting WAL directory at /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175691491353-13614-0/minicluster-data/ts-1/wal/wals/bd97489579bf4605b3737c2fc0bc5e01
I20260923 15:02:07.570986 21641 tablet_bootstrap.cc:492] T 903409fdfec246f88f037d3420c1a942 P f2f18bf473b04a23a122a756e8f646a3: No bootstrap required, opened a new log
I20260923 15:02:07.571004 21637 tablet_bootstrap.cc:654] T 903409fdfec246f88f037d3420c1a942 P b3dff0a518044ddeb2e65d3fbfb0ed68: Neither blocks nor log segments found. Creating new log.
I20260923 15:02:07.571033 21641 ts_tablet_manager.cc:1397] T 903409fdfec246f88f037d3420c1a942 P f2f18bf473b04a23a122a756e8f646a3: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.000s
I20260923 15:02:07.571043 21661 ts_tablet_manager.cc:1939] T bd97489579bf4605b3737c2fc0bc5e01 P 21a714af281940b497ab06ba85aaa27d: Deleting consensus metadata
I20260923 15:02:07.571206 21641 raft_consensus.cc:348] T 903409fdfec246f88f037d3420c1a942 P f2f18bf473b04a23a122a756e8f646a3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2f18bf473b04a23a122a756e8f646a3" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 38767 } } peers { permanent_uuid: "21a714af281940b497ab06ba85aaa27d" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 46875 } } peers { permanent_uuid: "b3dff0a518044ddeb2e65d3fbfb0ed68" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 39525 } }
I20260923 15:02:07.571278 21641 raft_consensus.cc:374] T 903409fdfec246f88f037d3420c1a942 P f2f18bf473b04a23a122a756e8f646a3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:02:07.571301 21641 raft_consensus.cc:729] T 903409fdfec246f88f037d3420c1a942 P f2f18bf473b04a23a122a756e8f646a3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2f18bf473b04a23a122a756e8f646a3, State: Initialized, Role: FOLLOWER
I20260923 15:02:07.571352 21641 consensus_queue.cc:260] T 903409fdfec246f88f037d3420c1a942 P f2f18bf473b04a23a122a756e8f646a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2f18bf473b04a23a122a756e8f646a3" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 38767 } } peers { permanent_uuid: "21a714af281940b497ab06ba85aaa27d" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 46875 } } peers { permanent_uuid: "b3dff0a518044ddeb2e65d3fbfb0ed68" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 39525 } }
I20260923 15:02:07.571455 21641 ts_tablet_manager.cc:1428] T 903409fdfec246f88f037d3420c1a942 P f2f18bf473b04a23a122a756e8f646a3: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260923 15:02:07.571673 21660 ts_tablet_manager.cc:1905] T bd97489579bf4605b3737c2fc0bc5e01 P f2f18bf473b04a23a122a756e8f646a3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 15:02:07.572171 21110 catalog_manager.cc:4841] TS 21a714af281940b497ab06ba85aaa27d (127.13.75.130:46875): tablet bd97489579bf4605b3737c2fc0bc5e01 (table test-table [id=defdea7049b243bf919fbbfa4bd0b265]) successfully deleted
I20260923 15:02:07.572405 21403 tablet_service.cc:1467] Processing CreateTablet for tablet 903409fdfec246f88f037d3420c1a942 (DEFAULT_TABLE table=test-table [id=c68093850b9b42aea4a70ae4a0f355c0]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 15:02:07.572530 21403 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 903409fdfec246f88f037d3420c1a942. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:02:07.573928 21638 tablet_bootstrap.cc:492] T 903409fdfec246f88f037d3420c1a942 P 21a714af281940b497ab06ba85aaa27d: Bootstrap starting.
I20260923 15:02:07.574048 21660 ts_tablet_manager.cc:1918] T bd97489579bf4605b3737c2fc0bc5e01 P f2f18bf473b04a23a122a756e8f646a3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 15:02:07.574136 21660 log.cc:1196] T bd97489579bf4605b3737c2fc0bc5e01 P f2f18bf473b04a23a122a756e8f646a3: Deleting WAL directory at /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175691491353-13614-0/minicluster-data/ts-0/wal/wals/bd97489579bf4605b3737c2fc0bc5e01
I20260923 15:02:07.574395 21638 tablet_bootstrap.cc:654] T 903409fdfec246f88f037d3420c1a942 P 21a714af281940b497ab06ba85aaa27d: Neither blocks nor log segments found. Creating new log.
I20260923 15:02:07.574435 21660 ts_tablet_manager.cc:1939] T bd97489579bf4605b3737c2fc0bc5e01 P f2f18bf473b04a23a122a756e8f646a3: Deleting consensus metadata
I20260923 15:02:07.570523 21662 raft_consensus.cc:2229] T bd97489579bf4605b3737c2fc0bc5e01 P b3dff0a518044ddeb2e65d3fbfb0ed68 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 15:02:07.574919 21112 catalog_manager.cc:4841] TS f2f18bf473b04a23a122a756e8f646a3 (127.13.75.129:38767): tablet bd97489579bf4605b3737c2fc0bc5e01 (table test-table [id=defdea7049b243bf919fbbfa4bd0b265]) successfully deleted
I20260923 15:02:07.575146 21662 raft_consensus.cc:2258] T bd97489579bf4605b3737c2fc0bc5e01 P b3dff0a518044ddeb2e65d3fbfb0ed68 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 15:02:07.575192 21638 tablet_bootstrap.cc:492] T 903409fdfec246f88f037d3420c1a942 P 21a714af281940b497ab06ba85aaa27d: No bootstrap required, opened a new log
I20260923 15:02:07.575240 21638 ts_tablet_manager.cc:1397] T 903409fdfec246f88f037d3420c1a942 P 21a714af281940b497ab06ba85aaa27d: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260923 15:02:07.575520 21638 raft_consensus.cc:348] T 903409fdfec246f88f037d3420c1a942 P 21a714af281940b497ab06ba85aaa27d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2f18bf473b04a23a122a756e8f646a3" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 38767 } } peers { permanent_uuid: "21a714af281940b497ab06ba85aaa27d" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 46875 } } peers { permanent_uuid: "b3dff0a518044ddeb2e65d3fbfb0ed68" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 39525 } }
I20260923 15:02:07.575619 21638 raft_consensus.cc:374] T 903409fdfec246f88f037d3420c1a942 P 21a714af281940b497ab06ba85aaa27d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:02:07.575644 21638 raft_consensus.cc:729] T 903409fdfec246f88f037d3420c1a942 P 21a714af281940b497ab06ba85aaa27d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 21a714af281940b497ab06ba85aaa27d, State: Initialized, Role: FOLLOWER
I20260923 15:02:07.575702 21638 consensus_queue.cc:260] T 903409fdfec246f88f037d3420c1a942 P 21a714af281940b497ab06ba85aaa27d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2f18bf473b04a23a122a756e8f646a3" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 38767 } } peers { permanent_uuid: "21a714af281940b497ab06ba85aaa27d" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 46875 } } peers { permanent_uuid: "b3dff0a518044ddeb2e65d3fbfb0ed68" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 39525 } }
I20260923 15:02:07.575809 21638 ts_tablet_manager.cc:1428] T 903409fdfec246f88f037d3420c1a942 P 21a714af281940b497ab06ba85aaa27d: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260923 15:02:07.575999 21637 tablet_bootstrap.cc:492] T 903409fdfec246f88f037d3420c1a942 P b3dff0a518044ddeb2e65d3fbfb0ed68: No bootstrap required, opened a new log
I20260923 15:02:07.576046 21637 ts_tablet_manager.cc:1397] T 903409fdfec246f88f037d3420c1a942 P b3dff0a518044ddeb2e65d3fbfb0ed68: Time spent bootstrapping tablet: real 0.006s user 0.000s sys 0.001s
I20260923 15:02:07.576175 21637 raft_consensus.cc:348] T 903409fdfec246f88f037d3420c1a942 P b3dff0a518044ddeb2e65d3fbfb0ed68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2f18bf473b04a23a122a756e8f646a3" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 38767 } } peers { permanent_uuid: "21a714af281940b497ab06ba85aaa27d" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 46875 } } peers { permanent_uuid: "b3dff0a518044ddeb2e65d3fbfb0ed68" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 39525 } }
I20260923 15:02:07.576223 21637 raft_consensus.cc:374] T 903409fdfec246f88f037d3420c1a942 P b3dff0a518044ddeb2e65d3fbfb0ed68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:02:07.576237 21637 raft_consensus.cc:729] T 903409fdfec246f88f037d3420c1a942 P b3dff0a518044ddeb2e65d3fbfb0ed68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3dff0a518044ddeb2e65d3fbfb0ed68, State: Initialized, Role: FOLLOWER
I20260923 15:02:07.576303 21637 consensus_queue.cc:260] T 903409fdfec246f88f037d3420c1a942 P b3dff0a518044ddeb2e65d3fbfb0ed68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2f18bf473b04a23a122a756e8f646a3" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 38767 } } peers { permanent_uuid: "21a714af281940b497ab06ba85aaa27d" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 46875 } } peers { permanent_uuid: "b3dff0a518044ddeb2e65d3fbfb0ed68" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 39525 } }
I20260923 15:02:07.576431 21637 ts_tablet_manager.cc:1428] T 903409fdfec246f88f037d3420c1a942 P b3dff0a518044ddeb2e65d3fbfb0ed68: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260923 15:02:07.576882 21662 ts_tablet_manager.cc:1905] T bd97489579bf4605b3737c2fc0bc5e01 P b3dff0a518044ddeb2e65d3fbfb0ed68: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 15:02:07.578547 21662 ts_tablet_manager.cc:1918] T bd97489579bf4605b3737c2fc0bc5e01 P b3dff0a518044ddeb2e65d3fbfb0ed68: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 15:02:07.578677 21662 log.cc:1196] T bd97489579bf4605b3737c2fc0bc5e01 P b3dff0a518044ddeb2e65d3fbfb0ed68: Deleting WAL directory at /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175691491353-13614-0/minicluster-data/ts-2/wal/wals/bd97489579bf4605b3737c2fc0bc5e01
I20260923 15:02:07.579027 21662 ts_tablet_manager.cc:1939] T bd97489579bf4605b3737c2fc0bc5e01 P b3dff0a518044ddeb2e65d3fbfb0ed68: Deleting consensus metadata
I20260923 15:02:07.579559 21113 catalog_manager.cc:4841] TS b3dff0a518044ddeb2e65d3fbfb0ed68 (127.13.75.131:39525): tablet bd97489579bf4605b3737c2fc0bc5e01 (table test-table [id=defdea7049b243bf919fbbfa4bd0b265]) successfully deleted
I20260923 15:02:07.606244 21643 raft_consensus.cc:482] T 903409fdfec246f88f037d3420c1a942 P 21a714af281940b497ab06ba85aaa27d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 15:02:07.606374 21643 raft_consensus.cc:504] T 903409fdfec246f88f037d3420c1a942 P 21a714af281940b497ab06ba85aaa27d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2f18bf473b04a23a122a756e8f646a3" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 38767 } } peers { permanent_uuid: "21a714af281940b497ab06ba85aaa27d" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 46875 } } peers { permanent_uuid: "b3dff0a518044ddeb2e65d3fbfb0ed68" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 39525 } }
I20260923 15:02:07.606649 21643 leader_election.cc:288] T 903409fdfec246f88f037d3420c1a942 P 21a714af281940b497ab06ba85aaa27d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f2f18bf473b04a23a122a756e8f646a3 (127.13.75.129:38767), b3dff0a518044ddeb2e65d3fbfb0ed68 (127.13.75.131:39525)
I20260923 15:02:07.606678 21642 raft_consensus.cc:482] T 903409fdfec246f88f037d3420c1a942 P b3dff0a518044ddeb2e65d3fbfb0ed68 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 15:02:07.606741 21642 raft_consensus.cc:504] T 903409fdfec246f88f037d3420c1a942 P b3dff0a518044ddeb2e65d3fbfb0ed68 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2f18bf473b04a23a122a756e8f646a3" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 38767 } } peers { permanent_uuid: "21a714af281940b497ab06ba85aaa27d" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 46875 } } peers { permanent_uuid: "b3dff0a518044ddeb2e65d3fbfb0ed68" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 39525 } }
I20260923 15:02:07.606913 21283 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "903409fdfec246f88f037d3420c1a942" candidate_uuid: "21a714af281940b497ab06ba85aaa27d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2f18bf473b04a23a122a756e8f646a3" is_pre_election: true
I20260923 15:02:07.607002 21283 raft_consensus.cc:2454] T 903409fdfec246f88f037d3420c1a942 P f2f18bf473b04a23a122a756e8f646a3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 21a714af281940b497ab06ba85aaa27d in term 0.
I20260923 15:02:07.607023 21563 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "903409fdfec246f88f037d3420c1a942" candidate_uuid: "21a714af281940b497ab06ba85aaa27d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3dff0a518044ddeb2e65d3fbfb0ed68" is_pre_election: true
I20260923 15:02:07.607108 21563 raft_consensus.cc:2454] T 903409fdfec246f88f037d3420c1a942 P b3dff0a518044ddeb2e65d3fbfb0ed68 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 21a714af281940b497ab06ba85aaa27d in term 0.
I20260923 15:02:07.607175 21356 leader_election.cc:302] T 903409fdfec246f88f037d3420c1a942 P 21a714af281940b497ab06ba85aaa27d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 21a714af281940b497ab06ba85aaa27d, f2f18bf473b04a23a122a756e8f646a3; no voters:
I20260923 15:02:07.607367 21643 raft_consensus.cc:2781] T 903409fdfec246f88f037d3420c1a942 P 21a714af281940b497ab06ba85aaa27d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 15:02:07.607416 21643 raft_consensus.cc:482] T 903409fdfec246f88f037d3420c1a942 P 21a714af281940b497ab06ba85aaa27d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 15:02:07.607443 21643 raft_consensus.cc:3037] T 903409fdfec246f88f037d3420c1a942 P 21a714af281940b497ab06ba85aaa27d [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:02:07.607113 21642 leader_election.cc:288] T 903409fdfec246f88f037d3420c1a942 P b3dff0a518044ddeb2e65d3fbfb0ed68 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f2f18bf473b04a23a122a756e8f646a3 (127.13.75.129:38767), 21a714af281940b497ab06ba85aaa27d (127.13.75.130:46875)
I20260923 15:02:07.608407 21643 raft_consensus.cc:504] T 903409fdfec246f88f037d3420c1a942 P 21a714af281940b497ab06ba85aaa27d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2f18bf473b04a23a122a756e8f646a3" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 38767 } } peers { permanent_uuid: "21a714af281940b497ab06ba85aaa27d" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 46875 } } peers { permanent_uuid: "b3dff0a518044ddeb2e65d3fbfb0ed68" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 39525 } }
I20260923 15:02:07.608542 21643 leader_election.cc:288] T 903409fdfec246f88f037d3420c1a942 P 21a714af281940b497ab06ba85aaa27d [CANDIDATE]: Term 1 election: Requested vote from peers f2f18bf473b04a23a122a756e8f646a3 (127.13.75.129:38767), b3dff0a518044ddeb2e65d3fbfb0ed68 (127.13.75.131:39525)
I20260923 15:02:07.608736 21283 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "903409fdfec246f88f037d3420c1a942" candidate_uuid: "21a714af281940b497ab06ba85aaa27d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2f18bf473b04a23a122a756e8f646a3"
I20260923 15:02:07.608806 21283 raft_consensus.cc:3037] T 903409fdfec246f88f037d3420c1a942 P f2f18bf473b04a23a122a756e8f646a3 [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:02:07.609617 21283 raft_consensus.cc:2454] T 903409fdfec246f88f037d3420c1a942 P f2f18bf473b04a23a122a756e8f646a3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 21a714af281940b497ab06ba85aaa27d in term 1.
I20260923 15:02:07.609797 21356 leader_election.cc:302] T 903409fdfec246f88f037d3420c1a942 P 21a714af281940b497ab06ba85aaa27d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 21a714af281940b497ab06ba85aaa27d, f2f18bf473b04a23a122a756e8f646a3; no voters:
I20260923 15:02:07.609871 21643 raft_consensus.cc:2781] T 903409fdfec246f88f037d3420c1a942 P 21a714af281940b497ab06ba85aaa27d [term 1 FOLLOWER]: Leader election won for term 1
I20260923 15:02:07.609905 21643 raft_consensus.cc:686] T 903409fdfec246f88f037d3420c1a942 P 21a714af281940b497ab06ba85aaa27d [term 1 LEADER]: Becoming Leader. State: Replica: 21a714af281940b497ab06ba85aaa27d, State: Running, Role: LEADER
I20260923 15:02:07.609962 21643 consensus_queue.cc:237] T 903409fdfec246f88f037d3420c1a942 P 21a714af281940b497ab06ba85aaa27d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2f18bf473b04a23a122a756e8f646a3" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 38767 } } peers { permanent_uuid: "21a714af281940b497ab06ba85aaa27d" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 46875 } } peers { permanent_uuid: "b3dff0a518044ddeb2e65d3fbfb0ed68" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 39525 } }
I20260923 15:02:07.609871 21563 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "903409fdfec246f88f037d3420c1a942" candidate_uuid: "21a714af281940b497ab06ba85aaa27d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3dff0a518044ddeb2e65d3fbfb0ed68"
I20260923 15:02:07.610039 21563 raft_consensus.cc:3037] T 903409fdfec246f88f037d3420c1a942 P b3dff0a518044ddeb2e65d3fbfb0ed68 [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:02:07.611505 21563 raft_consensus.cc:2454] T 903409fdfec246f88f037d3420c1a942 P b3dff0a518044ddeb2e65d3fbfb0ed68 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 21a714af281940b497ab06ba85aaa27d in term 1.
I20260923 15:02:07.611519 21636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:07.607733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.131:47765 (local address 127.13.75.129:38767)
0923 15:02:07.607775 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0923 15:02:07.607776 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 15:02:07.607789 (+ 13us) server_negotiation.cc:407] Connection header received
0923 15:02:07.607993 (+ 204us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:02:07.607994 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:02:07.608005 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:02:07.608064 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:02:07.608293 (+ 229us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:07.609020 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:07.610535 (+ 1515us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:07.611141 (+ 606us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:07.611193 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:07.611203 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0923 15:02:07.611411 (+ 208us) server_negotiation.cc:299] Negotiation successful
0923 15:02:07.611448 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260923 15:02:07.611799 21666 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:07.607541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.129:38767 (local address 127.13.75.131:47765)
0923 15:02:07.607733 (+ 192us) negotiation.cc:107] Waiting for socket to connect
0923 15:02:07.607744 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 15:02:07.607805 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:02:07.608120 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:02:07.608121 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:02:07.608123 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:02:07.608211 (+ 88us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:07.608213 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:07.609156 (+ 943us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:07.609157 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:07.610429 (+ 1272us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:07.610434 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:07.611262 (+ 828us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:07.611263 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:07.611335 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:07.611338 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 15:02:07.611625 (+ 287us) client_negotiation.cc:240] Negotiation successful
0923 15:02:07.611680 (+ 55us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":177,"thread_start_us":98,"threads_started":1}
I20260923 15:02:07.612044 21283 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "903409fdfec246f88f037d3420c1a942" candidate_uuid: "b3dff0a518044ddeb2e65d3fbfb0ed68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2f18bf473b04a23a122a756e8f646a3" is_pre_election: true
I20260923 15:02:07.612150 21283 raft_consensus.cc:2379] T 903409fdfec246f88f037d3420c1a942 P f2f18bf473b04a23a122a756e8f646a3 [term 1 FOLLOWER]: Leader pre-election vote request: Denying vote to candidate b3dff0a518044ddeb2e65d3fbfb0ed68 in current term 1: Already voted for candidate 21a714af281940b497ab06ba85aaa27d in this term.
I20260923 15:02:07.612071 21126 catalog_manager.cc:5495] T 903409fdfec246f88f037d3420c1a942 P 21a714af281940b497ab06ba85aaa27d reported cstate change: term changed from 0 to 1, leader changed from <none> to 21a714af281940b497ab06ba85aaa27d (127.13.75.130). New cstate: current_term: 1 leader_uuid: "21a714af281940b497ab06ba85aaa27d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2f18bf473b04a23a122a756e8f646a3" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 38767 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "21a714af281940b497ab06ba85aaa27d" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 46875 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b3dff0a518044ddeb2e65d3fbfb0ed68" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 39525 } health_report { overall_health: UNKNOWN } } }
I20260923 15:02:07.613113 21502 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:07.607271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.130:46875 (local address 127.13.75.131:57415)
0923 15:02:07.607335 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0923 15:02:07.607347 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 15:02:07.607396 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:02:07.608155 (+ 759us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:02:07.608156 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:02:07.608157 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:02:07.608219 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:07.608220 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:07.610463 (+ 2243us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:07.610464 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:07.611608 (+ 1144us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:07.611611 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:07.612822 (+ 1211us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:07.612823 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:07.612916 (+ 93us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:07.612921 (+ 5us) client_negotiation.cc:769] Sending connection context
0923 15:02:07.612973 (+ 52us) client_negotiation.cc:240] Negotiation successful
0923 15:02:07.613029 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44}
I20260923 15:02:07.613335 21635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:07.607297 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.131:57415 (local address 127.13.75.130:46875)
0923 15:02:07.607342 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0923 15:02:07.607344 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:02:07.607984 (+ 640us) server_negotiation.cc:407] Connection header received
0923 15:02:07.608011 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:02:07.608012 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:02:07.608022 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:02:07.608070 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:02:07.608291 (+ 221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:07.609510 (+ 1219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:07.612143 (+ 2633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:07.612709 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:07.612776 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:07.612786 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0923 15:02:07.613228 (+ 442us) server_negotiation.cc:299] Negotiation successful
0923 15:02:07.613273 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260923 15:02:07.613611 21423 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "903409fdfec246f88f037d3420c1a942" candidate_uuid: "b3dff0a518044ddeb2e65d3fbfb0ed68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21a714af281940b497ab06ba85aaa27d" is_pre_election: true
I20260923 15:02:07.613948 21494 leader_election.cc:302] T 903409fdfec246f88f037d3420c1a942 P b3dff0a518044ddeb2e65d3fbfb0ed68 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: b3dff0a518044ddeb2e65d3fbfb0ed68; no voters: 21a714af281940b497ab06ba85aaa27d, f2f18bf473b04a23a122a756e8f646a3
I20260923 15:02:07.614094 21642 raft_consensus.cc:2726] T 903409fdfec246f88f037d3420c1a942 P b3dff0a518044ddeb2e65d3fbfb0ed68 [term 1 FOLLOWER]: Leader pre-election lost for term 1. Reason: could not achieve majority
W20260923 15:02:07.636828 21610 tablet.cc:2369] T 903409fdfec246f88f037d3420c1a942 P b3dff0a518044ddeb2e65d3fbfb0ed68: Can't schedule compaction. Clean time has not been advanced past its initial value.
Sep 23 15:02:07 dist-test-slave-cnb0 krb5kdc[21081](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175726, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.75.190@KRBTEST.COM for oryx/127.13.75.190@KRBTEST.COM
I20260923 15:02:07.652804 21673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:07.643792 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:40515 (local address 127.0.0.1:35140)
0923 15:02:07.643967 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0923 15:02:07.643980 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0923 15:02:07.644053 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:02:07.644260 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:02:07.644261 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:02:07.644581 (+ 320us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:02:07.644637 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:07.644639 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:07.645619 (+ 980us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:07.645621 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:07.646170 (+ 549us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:07.646172 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:07.646669 (+ 497us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:07.646670 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:07.646722 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:07.647424 (+ 702us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:02:07.647429 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:02:07.649168 (+ 1739us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:02:07.651202 (+ 2034us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:02:07.651213 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:02:07.651215 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:02:07.651567 (+ 352us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:02:07.651832 (+ 265us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:02:07.651833 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:02:07.651833 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:02:07.651896 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:02:07.652209 (+ 313us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:02:07.652217 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:02:07.652289 (+ 72us) client_negotiation.cc:769] Sending connection context
0923 15:02:07.652497 (+ 208us) client_negotiation.cc:240] Negotiation successful
0923 15:02:07.652716 (+ 219us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":108,"threads_started":1}
I20260923 15:02:07.652922 21175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:07.643917 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35140 (local address 127.13.75.190:40515)
0923 15:02:07.643979 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0923 15:02:07.643981 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:02:07.644107 (+ 126us) server_negotiation.cc:407] Connection header received
0923 15:02:07.644139 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:02:07.644140 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:02:07.644152 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:02:07.644219 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:02:07.644750 (+ 531us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:07.645505 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:07.646281 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:07.646569 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:07.646653 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:07.649287 (+ 2634us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:02:07.649289 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:02:07.649289 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:02:07.649299 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:02:07.651078 (+ 1779us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:02:07.651689 (+ 611us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:02:07.651691 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:02:07.651691 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:02:07.651762 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:02:07.651964 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:02:07.651965 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:02:07.651966 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:02:07.652145 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:02:07.652216 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0923 15:02:07.652590 (+ 374us) server_negotiation.cc:299] Negotiation successful
0923 15:02:07.652804 (+ 214us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":112,"server-negotiator.queue_time_us":36}
I20260923 15:02:07.655377 21126 catalog_manager.cc:1908] Servicing CreateTable request from {username='oryx', principal='oryx/127.13.75.190@KRBTEST.COM'} at 127.0.0.1:35140:
name: "kudu_system.kudu_transactions"
schema {
columns {
name: "txn_id"
type: INT64
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "entry_type"
type: INT8
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "identifier"
type: STRING
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "metadata"
type: STRING
is_key: false
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
rows: "<redacted>""\006\001\000\000\000\000\000\000\000\000\007\001@B\017\000\000\000\000\000"
indirect_data: "<redacted>"""
}
partition_schema {
range_schema {
columns {
name: "txn_id"
}
}
}
table_type: TXN_STATUS_TABLE
W20260923 15:02:07.655649 21126 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table kudu_system.kudu_transactions in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 15:02:07.657490 21543 tablet_service.cc:1467] Processing CreateTablet for tablet b0c37bbe466240e582b455900daac8d6 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=0548ced6fd654a2f937ca99313b9cc98]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260923 15:02:07.657496 21403 tablet_service.cc:1467] Processing CreateTablet for tablet b0c37bbe466240e582b455900daac8d6 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=0548ced6fd654a2f937ca99313b9cc98]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260923 15:02:07.657508 21263 tablet_service.cc:1467] Processing CreateTablet for tablet b0c37bbe466240e582b455900daac8d6 (TXN_STATUS_TABLE table=kudu_system.kudu_transactions [id=0548ced6fd654a2f937ca99313b9cc98]), partition=RANGE (txn_id) PARTITION 0 <= VALUES < 1000000
I20260923 15:02:07.657681 21403 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b0c37bbe466240e582b455900daac8d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:02:07.657680 21263 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b0c37bbe466240e582b455900daac8d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:02:07.657681 21543 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet b0c37bbe466240e582b455900daac8d6. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:02:07.659471 21638 tablet_bootstrap.cc:492] T b0c37bbe466240e582b455900daac8d6 P 21a714af281940b497ab06ba85aaa27d: Bootstrap starting.
I20260923 15:02:07.659998 21638 tablet_bootstrap.cc:654] T b0c37bbe466240e582b455900daac8d6 P 21a714af281940b497ab06ba85aaa27d: Neither blocks nor log segments found. Creating new log.
I20260923 15:02:07.661037 21641 tablet_bootstrap.cc:492] T b0c37bbe466240e582b455900daac8d6 P f2f18bf473b04a23a122a756e8f646a3: Bootstrap starting.
I20260923 15:02:07.661533 21638 tablet_bootstrap.cc:492] T b0c37bbe466240e582b455900daac8d6 P 21a714af281940b497ab06ba85aaa27d: No bootstrap required, opened a new log
I20260923 15:02:07.661566 21641 tablet_bootstrap.cc:654] T b0c37bbe466240e582b455900daac8d6 P f2f18bf473b04a23a122a756e8f646a3: Neither blocks nor log segments found. Creating new log.
I20260923 15:02:07.661600 21638 ts_tablet_manager.cc:1397] T b0c37bbe466240e582b455900daac8d6 P 21a714af281940b497ab06ba85aaa27d: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 15:02:07.661792 21638 raft_consensus.cc:348] T b0c37bbe466240e582b455900daac8d6 P 21a714af281940b497ab06ba85aaa27d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2f18bf473b04a23a122a756e8f646a3" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 38767 } } peers { permanent_uuid: "21a714af281940b497ab06ba85aaa27d" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 46875 } } peers { permanent_uuid: "b3dff0a518044ddeb2e65d3fbfb0ed68" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 39525 } }
I20260923 15:02:07.661926 21638 raft_consensus.cc:374] T b0c37bbe466240e582b455900daac8d6 P 21a714af281940b497ab06ba85aaa27d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:02:07.661957 21638 raft_consensus.cc:729] T b0c37bbe466240e582b455900daac8d6 P 21a714af281940b497ab06ba85aaa27d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 21a714af281940b497ab06ba85aaa27d, State: Initialized, Role: FOLLOWER
I20260923 15:02:07.662022 21638 consensus_queue.cc:260] T b0c37bbe466240e582b455900daac8d6 P 21a714af281940b497ab06ba85aaa27d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2f18bf473b04a23a122a756e8f646a3" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 38767 } } peers { permanent_uuid: "21a714af281940b497ab06ba85aaa27d" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 46875 } } peers { permanent_uuid: "b3dff0a518044ddeb2e65d3fbfb0ed68" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 39525 } }
I20260923 15:02:07.662142 21638 ts_tablet_manager.cc:1428] T b0c37bbe466240e582b455900daac8d6 P 21a714af281940b497ab06ba85aaa27d: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260923 15:02:07.662156 21647 tablet_replica.cc:439] T b0c37bbe466240e582b455900daac8d6 P 21a714af281940b497ab06ba85aaa27d: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2f18bf473b04a23a122a756e8f646a3" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 38767 } } peers { permanent_uuid: "21a714af281940b497ab06ba85aaa27d" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 46875 } } peers { permanent_uuid: "b3dff0a518044ddeb2e65d3fbfb0ed68" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 39525 } } }
I20260923 15:02:07.662245 21647 tablet_replica.cc:442] T b0c37bbe466240e582b455900daac8d6 P 21a714af281940b497ab06ba85aaa27d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260923 15:02:07.662981 21641 tablet_bootstrap.cc:492] T b0c37bbe466240e582b455900daac8d6 P f2f18bf473b04a23a122a756e8f646a3: No bootstrap required, opened a new log
I20260923 15:02:07.663062 21641 ts_tablet_manager.cc:1397] T b0c37bbe466240e582b455900daac8d6 P f2f18bf473b04a23a122a756e8f646a3: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 15:02:07.663239 21641 raft_consensus.cc:348] T b0c37bbe466240e582b455900daac8d6 P f2f18bf473b04a23a122a756e8f646a3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2f18bf473b04a23a122a756e8f646a3" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 38767 } } peers { permanent_uuid: "21a714af281940b497ab06ba85aaa27d" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 46875 } } peers { permanent_uuid: "b3dff0a518044ddeb2e65d3fbfb0ed68" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 39525 } }
I20260923 15:02:07.663342 21641 raft_consensus.cc:374] T b0c37bbe466240e582b455900daac8d6 P f2f18bf473b04a23a122a756e8f646a3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:02:07.663378 21641 raft_consensus.cc:729] T b0c37bbe466240e582b455900daac8d6 P f2f18bf473b04a23a122a756e8f646a3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2f18bf473b04a23a122a756e8f646a3, State: Initialized, Role: FOLLOWER
I20260923 15:02:07.663437 21641 consensus_queue.cc:260] T b0c37bbe466240e582b455900daac8d6 P f2f18bf473b04a23a122a756e8f646a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2f18bf473b04a23a122a756e8f646a3" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 38767 } } peers { permanent_uuid: "21a714af281940b497ab06ba85aaa27d" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 46875 } } peers { permanent_uuid: "b3dff0a518044ddeb2e65d3fbfb0ed68" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 39525 } }
I20260923 15:02:07.663553 21641 ts_tablet_manager.cc:1428] T b0c37bbe466240e582b455900daac8d6 P f2f18bf473b04a23a122a756e8f646a3: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260923 15:02:07.664860 21645 tablet_replica.cc:439] T b0c37bbe466240e582b455900daac8d6 P f2f18bf473b04a23a122a756e8f646a3: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2f18bf473b04a23a122a756e8f646a3" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 38767 } } peers { permanent_uuid: "21a714af281940b497ab06ba85aaa27d" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 46875 } } peers { permanent_uuid: "b3dff0a518044ddeb2e65d3fbfb0ed68" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 39525 } } }
I20260923 15:02:07.665010 21645 tablet_replica.cc:442] T b0c37bbe466240e582b455900daac8d6 P f2f18bf473b04a23a122a756e8f646a3: This TxnStatusTablet replica's current role is: FOLLOWER
I20260923 15:02:07.665721 21637 tablet_bootstrap.cc:492] T b0c37bbe466240e582b455900daac8d6 P b3dff0a518044ddeb2e65d3fbfb0ed68: Bootstrap starting.
I20260923 15:02:07.667721 21637 tablet_bootstrap.cc:654] T b0c37bbe466240e582b455900daac8d6 P b3dff0a518044ddeb2e65d3fbfb0ed68: Neither blocks nor log segments found. Creating new log.
I20260923 15:02:07.668640 21637 tablet_bootstrap.cc:492] T b0c37bbe466240e582b455900daac8d6 P b3dff0a518044ddeb2e65d3fbfb0ed68: No bootstrap required, opened a new log
I20260923 15:02:07.668732 21637 ts_tablet_manager.cc:1397] T b0c37bbe466240e582b455900daac8d6 P b3dff0a518044ddeb2e65d3fbfb0ed68: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.001s
I20260923 15:02:07.668902 21637 raft_consensus.cc:348] T b0c37bbe466240e582b455900daac8d6 P b3dff0a518044ddeb2e65d3fbfb0ed68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2f18bf473b04a23a122a756e8f646a3" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 38767 } } peers { permanent_uuid: "21a714af281940b497ab06ba85aaa27d" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 46875 } } peers { permanent_uuid: "b3dff0a518044ddeb2e65d3fbfb0ed68" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 39525 } }
I20260923 15:02:07.668975 21637 raft_consensus.cc:374] T b0c37bbe466240e582b455900daac8d6 P b3dff0a518044ddeb2e65d3fbfb0ed68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:02:07.669018 21637 raft_consensus.cc:729] T b0c37bbe466240e582b455900daac8d6 P b3dff0a518044ddeb2e65d3fbfb0ed68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3dff0a518044ddeb2e65d3fbfb0ed68, State: Initialized, Role: FOLLOWER
I20260923 15:02:07.669080 21637 consensus_queue.cc:260] T b0c37bbe466240e582b455900daac8d6 P b3dff0a518044ddeb2e65d3fbfb0ed68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2f18bf473b04a23a122a756e8f646a3" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 38767 } } peers { permanent_uuid: "21a714af281940b497ab06ba85aaa27d" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 46875 } } peers { permanent_uuid: "b3dff0a518044ddeb2e65d3fbfb0ed68" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 39525 } }
I20260923 15:02:07.669219 21637 ts_tablet_manager.cc:1428] T b0c37bbe466240e582b455900daac8d6 P b3dff0a518044ddeb2e65d3fbfb0ed68: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260923 15:02:07.669359 21642 tablet_replica.cc:439] T b0c37bbe466240e582b455900daac8d6 P b3dff0a518044ddeb2e65d3fbfb0ed68: TxnStatusTablet state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2f18bf473b04a23a122a756e8f646a3" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 38767 } } peers { permanent_uuid: "21a714af281940b497ab06ba85aaa27d" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 46875 } } peers { permanent_uuid: "b3dff0a518044ddeb2e65d3fbfb0ed68" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 39525 } } }
I20260923 15:02:07.669437 21642 tablet_replica.cc:442] T b0c37bbe466240e582b455900daac8d6 P b3dff0a518044ddeb2e65d3fbfb0ed68: This TxnStatusTablet replica's current role is: FOLLOWER
I20260923 15:02:07.716375 21642 raft_consensus.cc:482] T b0c37bbe466240e582b455900daac8d6 P b3dff0a518044ddeb2e65d3fbfb0ed68 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 15:02:07.716502 21642 raft_consensus.cc:504] T b0c37bbe466240e582b455900daac8d6 P b3dff0a518044ddeb2e65d3fbfb0ed68 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2f18bf473b04a23a122a756e8f646a3" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 38767 } } peers { permanent_uuid: "21a714af281940b497ab06ba85aaa27d" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 46875 } } peers { permanent_uuid: "b3dff0a518044ddeb2e65d3fbfb0ed68" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 39525 } }
I20260923 15:02:07.716691 21642 leader_election.cc:288] T b0c37bbe466240e582b455900daac8d6 P b3dff0a518044ddeb2e65d3fbfb0ed68 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f2f18bf473b04a23a122a756e8f646a3 (127.13.75.129:38767), 21a714af281940b497ab06ba85aaa27d (127.13.75.130:46875)
I20260923 15:02:07.716903 21283 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b0c37bbe466240e582b455900daac8d6" candidate_uuid: "b3dff0a518044ddeb2e65d3fbfb0ed68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2f18bf473b04a23a122a756e8f646a3" is_pre_election: true
I20260923 15:02:07.717006 21283 raft_consensus.cc:2454] T b0c37bbe466240e582b455900daac8d6 P f2f18bf473b04a23a122a756e8f646a3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3dff0a518044ddeb2e65d3fbfb0ed68 in term 0.
I20260923 15:02:07.716974 21423 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b0c37bbe466240e582b455900daac8d6" candidate_uuid: "b3dff0a518044ddeb2e65d3fbfb0ed68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21a714af281940b497ab06ba85aaa27d" is_pre_election: true
I20260923 15:02:07.717113 21423 raft_consensus.cc:2454] T b0c37bbe466240e582b455900daac8d6 P 21a714af281940b497ab06ba85aaa27d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate b3dff0a518044ddeb2e65d3fbfb0ed68 in term 0.
I20260923 15:02:07.717590 21496 leader_election.cc:302] T b0c37bbe466240e582b455900daac8d6 P b3dff0a518044ddeb2e65d3fbfb0ed68 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b3dff0a518044ddeb2e65d3fbfb0ed68, f2f18bf473b04a23a122a756e8f646a3; no voters:
I20260923 15:02:07.717725 21642 raft_consensus.cc:2781] T b0c37bbe466240e582b455900daac8d6 P b3dff0a518044ddeb2e65d3fbfb0ed68 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 15:02:07.717770 21642 raft_consensus.cc:482] T b0c37bbe466240e582b455900daac8d6 P b3dff0a518044ddeb2e65d3fbfb0ed68 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 15:02:07.717792 21642 raft_consensus.cc:3037] T b0c37bbe466240e582b455900daac8d6 P b3dff0a518044ddeb2e65d3fbfb0ed68 [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:02:07.718605 21642 raft_consensus.cc:504] T b0c37bbe466240e582b455900daac8d6 P b3dff0a518044ddeb2e65d3fbfb0ed68 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2f18bf473b04a23a122a756e8f646a3" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 38767 } } peers { permanent_uuid: "21a714af281940b497ab06ba85aaa27d" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 46875 } } peers { permanent_uuid: "b3dff0a518044ddeb2e65d3fbfb0ed68" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 39525 } }
I20260923 15:02:07.718772 21642 leader_election.cc:288] T b0c37bbe466240e582b455900daac8d6 P b3dff0a518044ddeb2e65d3fbfb0ed68 [CANDIDATE]: Term 1 election: Requested vote from peers f2f18bf473b04a23a122a756e8f646a3 (127.13.75.129:38767), 21a714af281940b497ab06ba85aaa27d (127.13.75.130:46875)
I20260923 15:02:07.718976 21283 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b0c37bbe466240e582b455900daac8d6" candidate_uuid: "b3dff0a518044ddeb2e65d3fbfb0ed68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2f18bf473b04a23a122a756e8f646a3"
I20260923 15:02:07.719079 21283 raft_consensus.cc:3037] T b0c37bbe466240e582b455900daac8d6 P f2f18bf473b04a23a122a756e8f646a3 [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:02:07.719054 21423 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "b0c37bbe466240e582b455900daac8d6" candidate_uuid: "b3dff0a518044ddeb2e65d3fbfb0ed68" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21a714af281940b497ab06ba85aaa27d"
I20260923 15:02:07.719172 21423 raft_consensus.cc:3037] T b0c37bbe466240e582b455900daac8d6 P 21a714af281940b497ab06ba85aaa27d [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:02:07.719769 21283 raft_consensus.cc:2454] T b0c37bbe466240e582b455900daac8d6 P f2f18bf473b04a23a122a756e8f646a3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3dff0a518044ddeb2e65d3fbfb0ed68 in term 1.
I20260923 15:02:07.719964 21496 leader_election.cc:302] T b0c37bbe466240e582b455900daac8d6 P b3dff0a518044ddeb2e65d3fbfb0ed68 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b3dff0a518044ddeb2e65d3fbfb0ed68, f2f18bf473b04a23a122a756e8f646a3; no voters:
I20260923 15:02:07.720103 21642 raft_consensus.cc:2781] T b0c37bbe466240e582b455900daac8d6 P b3dff0a518044ddeb2e65d3fbfb0ed68 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 15:02:07.720273 21642 raft_consensus.cc:686] T b0c37bbe466240e582b455900daac8d6 P b3dff0a518044ddeb2e65d3fbfb0ed68 [term 1 LEADER]: Becoming Leader. State: Replica: b3dff0a518044ddeb2e65d3fbfb0ed68, State: Running, Role: LEADER
I20260923 15:02:07.720398 21423 raft_consensus.cc:2454] T b0c37bbe466240e582b455900daac8d6 P 21a714af281940b497ab06ba85aaa27d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate b3dff0a518044ddeb2e65d3fbfb0ed68 in term 1.
I20260923 15:02:07.720431 21642 consensus_queue.cc:237] T b0c37bbe466240e582b455900daac8d6 P b3dff0a518044ddeb2e65d3fbfb0ed68 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2f18bf473b04a23a122a756e8f646a3" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 38767 } } peers { permanent_uuid: "21a714af281940b497ab06ba85aaa27d" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 46875 } } peers { permanent_uuid: "b3dff0a518044ddeb2e65d3fbfb0ed68" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 39525 } }
I20260923 15:02:07.720945 21677 tablet_replica.cc:439] T b0c37bbe466240e582b455900daac8d6 P b3dff0a518044ddeb2e65d3fbfb0ed68: TxnStatusTablet state changed. Reason: New leader b3dff0a518044ddeb2e65d3fbfb0ed68. Latest consensus state: current_term: 1 leader_uuid: "b3dff0a518044ddeb2e65d3fbfb0ed68" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2f18bf473b04a23a122a756e8f646a3" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 38767 } } peers { permanent_uuid: "21a714af281940b497ab06ba85aaa27d" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 46875 } } peers { permanent_uuid: "b3dff0a518044ddeb2e65d3fbfb0ed68" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 39525 } } }
I20260923 15:02:07.721068 21677 tablet_replica.cc:442] T b0c37bbe466240e582b455900daac8d6 P b3dff0a518044ddeb2e65d3fbfb0ed68: This TxnStatusTablet replica's current role is: LEADER
I20260923 15:02:07.721432 21679 txn_status_manager.cc:873] Waiting until node catch up with all replicated operations in previous term...
I20260923 15:02:07.721566 21679 txn_status_manager.cc:929] Loading transaction status metadata into memory...
I20260923 15:02:07.721503 21126 catalog_manager.cc:5495] T b0c37bbe466240e582b455900daac8d6 P b3dff0a518044ddeb2e65d3fbfb0ed68 reported cstate change: term changed from 0 to 1, leader changed from <none> to b3dff0a518044ddeb2e65d3fbfb0ed68 (127.13.75.131). New cstate: current_term: 1 leader_uuid: "b3dff0a518044ddeb2e65d3fbfb0ed68" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2f18bf473b04a23a122a756e8f646a3" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 38767 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "21a714af281940b497ab06ba85aaa27d" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 46875 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b3dff0a518044ddeb2e65d3fbfb0ed68" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 39525 } health_report { overall_health: HEALTHY } } }
I20260923 15:02:07.733597 21673 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:07.730443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.131:39525 (local address 127.0.0.1:56748)
0923 15:02:07.730532 (+ 89us) negotiation.cc:107] Waiting for socket to connect
0923 15:02:07.730551 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0923 15:02:07.730634 (+ 83us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:02:07.730846 (+ 212us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:02:07.730847 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:02:07.730849 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0923 15:02:07.730965 (+ 116us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:07.730968 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:07.731992 (+ 1024us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:07.731994 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:07.732598 (+ 604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:07.732600 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:07.732991 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:07.732992 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:07.733064 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:07.733085 (+ 21us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0923 15:02:07.733393 (+ 308us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0923 15:02:07.733404 (+ 11us) client_negotiation.cc:769] Sending connection context
0923 15:02:07.733465 (+ 61us) client_negotiation.cc:240] Negotiation successful
0923 15:02:07.733525 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":41}
I20260923 15:02:07.733582 21634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:07.730539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56748 (local address 127.13.75.131:39525)
0923 15:02:07.730606 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0923 15:02:07.730608 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:02:07.730618 (+ 10us) server_negotiation.cc:407] Connection header received
0923 15:02:07.730736 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:02:07.730737 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:02:07.730757 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:02:07.730841 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 15:02:07.731176 (+ 335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:07.731848 (+ 672us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:07.732704 (+ 856us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:07.732903 (+ 199us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:07.732980 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:07.733190 (+ 210us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 15:02:07.733303 (+ 113us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 15:02:07.733409 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0923 15:02:07.733471 (+ 62us) server_negotiation.cc:299] Negotiation successful
0923 15:02:07.733495 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260923 15:02:07.735363 21283 raft_consensus.cc:1261] T b0c37bbe466240e582b455900daac8d6 P f2f18bf473b04a23a122a756e8f646a3 [term 1 FOLLOWER]: Refusing update from remote peer b3dff0a518044ddeb2e65d3fbfb0ed68: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 15:02:07.735386 21423 raft_consensus.cc:1261] T b0c37bbe466240e582b455900daac8d6 P 21a714af281940b497ab06ba85aaa27d [term 1 FOLLOWER]: Refusing update from remote peer b3dff0a518044ddeb2e65d3fbfb0ed68: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 15:02:07.735616 21642 consensus_queue.cc:1035] T b0c37bbe466240e582b455900daac8d6 P b3dff0a518044ddeb2e65d3fbfb0ed68 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2f18bf473b04a23a122a756e8f646a3" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 38767 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 15:02:07.735793 21677 consensus_queue.cc:1035] T b0c37bbe466240e582b455900daac8d6 P b3dff0a518044ddeb2e65d3fbfb0ed68 [LEADER]: Connected to new peer: Peer: permanent_uuid: "21a714af281940b497ab06ba85aaa27d" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 15:02:07.736581 21645 tablet_replica.cc:439] T b0c37bbe466240e582b455900daac8d6 P f2f18bf473b04a23a122a756e8f646a3: TxnStatusTablet state changed. Reason: New leader b3dff0a518044ddeb2e65d3fbfb0ed68. Latest consensus state: current_term: 1 leader_uuid: "b3dff0a518044ddeb2e65d3fbfb0ed68" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2f18bf473b04a23a122a756e8f646a3" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 38767 } } peers { permanent_uuid: "21a714af281940b497ab06ba85aaa27d" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 46875 } } peers { permanent_uuid: "b3dff0a518044ddeb2e65d3fbfb0ed68" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 39525 } } }
I20260923 15:02:07.736689 21645 tablet_replica.cc:442] T b0c37bbe466240e582b455900daac8d6 P f2f18bf473b04a23a122a756e8f646a3: This TxnStatusTablet replica's current role is: FOLLOWER
I20260923 15:02:07.736733 21647 tablet_replica.cc:439] T b0c37bbe466240e582b455900daac8d6 P 21a714af281940b497ab06ba85aaa27d: TxnStatusTablet state changed. Reason: New leader b3dff0a518044ddeb2e65d3fbfb0ed68. Latest consensus state: current_term: 1 leader_uuid: "b3dff0a518044ddeb2e65d3fbfb0ed68" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2f18bf473b04a23a122a756e8f646a3" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 38767 } } peers { permanent_uuid: "21a714af281940b497ab06ba85aaa27d" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 46875 } } peers { permanent_uuid: "b3dff0a518044ddeb2e65d3fbfb0ed68" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 39525 } } }
I20260923 15:02:07.736819 21647 tablet_replica.cc:442] T b0c37bbe466240e582b455900daac8d6 P 21a714af281940b497ab06ba85aaa27d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260923 15:02:07.737902 21677 tablet_replica.cc:439] T b0c37bbe466240e582b455900daac8d6 P b3dff0a518044ddeb2e65d3fbfb0ed68: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b3dff0a518044ddeb2e65d3fbfb0ed68" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2f18bf473b04a23a122a756e8f646a3" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 38767 } } peers { permanent_uuid: "21a714af281940b497ab06ba85aaa27d" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 46875 } } peers { permanent_uuid: "b3dff0a518044ddeb2e65d3fbfb0ed68" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 39525 } } }
I20260923 15:02:07.738003 21677 tablet_replica.cc:442] T b0c37bbe466240e582b455900daac8d6 P b3dff0a518044ddeb2e65d3fbfb0ed68: This TxnStatusTablet replica's current role is: LEADER
I20260923 15:02:07.738255 21645 tablet_replica.cc:439] T b0c37bbe466240e582b455900daac8d6 P f2f18bf473b04a23a122a756e8f646a3: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b3dff0a518044ddeb2e65d3fbfb0ed68" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2f18bf473b04a23a122a756e8f646a3" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 38767 } } peers { permanent_uuid: "21a714af281940b497ab06ba85aaa27d" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 46875 } } peers { permanent_uuid: "b3dff0a518044ddeb2e65d3fbfb0ed68" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 39525 } } }
I20260923 15:02:07.738361 21645 tablet_replica.cc:442] T b0c37bbe466240e582b455900daac8d6 P f2f18bf473b04a23a122a756e8f646a3: This TxnStatusTablet replica's current role is: FOLLOWER
I20260923 15:02:07.744167 21677 tablet_replica.cc:439] T b0c37bbe466240e582b455900daac8d6 P b3dff0a518044ddeb2e65d3fbfb0ed68: TxnStatusTablet state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "b3dff0a518044ddeb2e65d3fbfb0ed68" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2f18bf473b04a23a122a756e8f646a3" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 38767 } } peers { permanent_uuid: "21a714af281940b497ab06ba85aaa27d" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 46875 } } peers { permanent_uuid: "b3dff0a518044ddeb2e65d3fbfb0ed68" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 39525 } } }
I20260923 15:02:07.744381 21677 tablet_replica.cc:442] T b0c37bbe466240e582b455900daac8d6 P b3dff0a518044ddeb2e65d3fbfb0ed68: This TxnStatusTablet replica's current role is: LEADER
I20260923 15:02:07.744772 21650 mvcc.cc:205] Tried to move back new op lower bound from 7332559780801437696 to 7332559780743401472. Current Snapshot: MvccSnapshot[applied={T|T < 7332559780801437696}]
I20260923 15:02:07.744872 21647 tablet_replica.cc:439] T b0c37bbe466240e582b455900daac8d6 P 21a714af281940b497ab06ba85aaa27d: TxnStatusTablet state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "b3dff0a518044ddeb2e65d3fbfb0ed68" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2f18bf473b04a23a122a756e8f646a3" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 38767 } } peers { permanent_uuid: "21a714af281940b497ab06ba85aaa27d" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 46875 } } peers { permanent_uuid: "b3dff0a518044ddeb2e65d3fbfb0ed68" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 39525 } } }
I20260923 15:02:07.744971 21647 tablet_replica.cc:442] T b0c37bbe466240e582b455900daac8d6 P 21a714af281940b497ab06ba85aaa27d: This TxnStatusTablet replica's current role is: FOLLOWER
I20260923 15:02:07.746198 21687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:07.742540 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.131:39525 (local address 127.13.75.130:43163)
0923 15:02:07.743068 (+ 528us) negotiation.cc:107] Waiting for socket to connect
0923 15:02:07.743080 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 15:02:07.743153 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:02:07.743309 (+ 156us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:02:07.743310 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:02:07.743312 (+ 2us) client_negotiation.cc:189] Negotiated authn=TOKEN
0923 15:02:07.743381 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:07.743383 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:07.744261 (+ 878us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:07.744263 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:07.745204 (+ 941us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:07.745207 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:07.745795 (+ 588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:07.745796 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:07.745841 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:07.745856 (+ 15us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0923 15:02:07.746065 (+ 209us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0923 15:02:07.746066 (+ 1us) client_negotiation.cc:769] Sending connection context
0923 15:02:07.746095 (+ 29us) client_negotiation.cc:240] Negotiation successful
0923 15:02:07.746135 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":458,"thread_start_us":85,"threads_started":1}
I20260923 15:02:07.746208 21634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:07.742657 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.130:43163 (local address 127.13.75.131:39525)
0923 15:02:07.742723 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0923 15:02:07.742726 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:02:07.743196 (+ 470us) server_negotiation.cc:407] Connection header received
0923 15:02:07.743228 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:02:07.743243 (+ 15us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:02:07.743257 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:02:07.743305 (+ 48us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 15:02:07.743468 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:07.744163 (+ 695us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:07.745311 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:07.745697 (+ 386us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:07.745740 (+ 43us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:07.745958 (+ 218us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 15:02:07.745999 (+ 41us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 15:02:07.746067 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0923 15:02:07.746106 (+ 39us) server_negotiation.cc:299] Negotiation successful
0923 15:02:07.746130 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260923 15:02:07.748939 21283 raft_consensus.cc:1261] T 903409fdfec246f88f037d3420c1a942 P f2f18bf473b04a23a122a756e8f646a3 [term 1 FOLLOWER]: Refusing update from remote peer 21a714af281940b497ab06ba85aaa27d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 15:02:07.749013 21563 raft_consensus.cc:1261] T 903409fdfec246f88f037d3420c1a942 P b3dff0a518044ddeb2e65d3fbfb0ed68 [term 1 FOLLOWER]: Refusing update from remote peer 21a714af281940b497ab06ba85aaa27d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 15:02:07.749229 21647 consensus_queue.cc:1035] T 903409fdfec246f88f037d3420c1a942 P 21a714af281940b497ab06ba85aaa27d [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3dff0a518044ddeb2e65d3fbfb0ed68" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 39525 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 15:02:07.749426 21647 consensus_queue.cc:1035] T 903409fdfec246f88f037d3420c1a942 P 21a714af281940b497ab06ba85aaa27d [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2f18bf473b04a23a122a756e8f646a3" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 38767 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 15:02:07.759300 21635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:07.756271 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.131:54225 (local address 127.13.75.130:46875)
0923 15:02:07.756311 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0923 15:02:07.756314 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:02:07.756355 (+ 41us) server_negotiation.cc:407] Connection header received
0923 15:02:07.756431 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:02:07.756432 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:02:07.756442 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:02:07.756490 (+ 48us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 15:02:07.756636 (+ 146us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:07.757388 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:07.758309 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:07.758628 (+ 319us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:07.758707 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:07.759004 (+ 297us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 15:02:07.759051 (+ 47us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 15:02:07.759106 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0923 15:02:07.759181 (+ 75us) server_negotiation.cc:299] Negotiation successful
0923 15:02:07.759208 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260923 15:02:07.759305 21615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:07.756225 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.130:46875 (local address 127.13.75.131:54225)
0923 15:02:07.756309 (+ 84us) negotiation.cc:107] Waiting for socket to connect
0923 15:02:07.756321 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 15:02:07.756370 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:02:07.756501 (+ 131us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:02:07.756502 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:02:07.756503 (+ 1us) client_negotiation.cc:189] Negotiated authn=TOKEN
0923 15:02:07.756564 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:07.756566 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:07.757528 (+ 962us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:07.757529 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:07.758212 (+ 683us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:07.758215 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:07.758706 (+ 491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:07.758707 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:07.758795 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:07.758820 (+ 25us) client_negotiation.cc:252] Sending TOKEN_EXCHANGE NegotiatePB request
0923 15:02:07.759120 (+ 300us) client_negotiation.cc:271] Received TOKEN_EXCHANGE NegotiatePB response
0923 15:02:07.759122 (+ 2us) client_negotiation.cc:769] Sending connection context
0923 15:02:07.759176 (+ 54us) client_negotiation.cc:240] Negotiation successful
0923 15:02:07.759228 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44}
I20260923 15:02:07.769585 21126 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35124:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 15:02:07.769860 21126 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-user', principal='test-user@KRBTEST.COM'} at 127.0.0.1:35124:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 15:02:07.770813 21126 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 385ef6dbf6774aa09aff37a6dcf6afd0: Sending DeleteTablet for 3 replicas of tablet 903409fdfec246f88f037d3420c1a942
I20260923 15:02:07.771147 21263 tablet_service.cc:1514] Processing DeleteTablet for tablet 903409fdfec246f88f037d3420c1a942 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 15:02:07 UTC) from {username='oryx', principal='oryx/127.13.75.190@KRBTEST.COM'} at 127.0.0.1:35798
I20260923 15:02:07.771173 21543 tablet_service.cc:1514] Processing DeleteTablet for tablet 903409fdfec246f88f037d3420c1a942 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 15:02:07 UTC) from {username='oryx', principal='oryx/127.13.75.190@KRBTEST.COM'} at 127.0.0.1:56734
I20260923 15:02:07.771268 21660 tablet_replica.cc:330] T 903409fdfec246f88f037d3420c1a942 P f2f18bf473b04a23a122a756e8f646a3: stopping tablet replica
I20260923 15:02:07.771289 21662 tablet_replica.cc:330] T 903409fdfec246f88f037d3420c1a942 P b3dff0a518044ddeb2e65d3fbfb0ed68: stopping tablet replica
I20260923 15:02:07.771293 21403 tablet_service.cc:1514] Processing DeleteTablet for tablet 903409fdfec246f88f037d3420c1a942 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 15:02:07 UTC) from {username='oryx', principal='oryx/127.13.75.190@KRBTEST.COM'} at 127.0.0.1:52954
I20260923 15:02:07.771327 21660 raft_consensus.cc:2229] T 903409fdfec246f88f037d3420c1a942 P f2f18bf473b04a23a122a756e8f646a3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 15:02:07.771350 21662 raft_consensus.cc:2229] T 903409fdfec246f88f037d3420c1a942 P b3dff0a518044ddeb2e65d3fbfb0ed68 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 15:02:07.771368 21660 raft_consensus.cc:2258] T 903409fdfec246f88f037d3420c1a942 P f2f18bf473b04a23a122a756e8f646a3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 15:02:07.771387 21662 raft_consensus.cc:2258] T 903409fdfec246f88f037d3420c1a942 P b3dff0a518044ddeb2e65d3fbfb0ed68 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 15:02:07.771446 21661 tablet_replica.cc:330] T 903409fdfec246f88f037d3420c1a942 P 21a714af281940b497ab06ba85aaa27d: stopping tablet replica
I20260923 15:02:07.771515 21661 raft_consensus.cc:2229] T 903409fdfec246f88f037d3420c1a942 P 21a714af281940b497ab06ba85aaa27d [term 1 LEADER]: Raft consensus shutting down.
I20260923 15:02:07.771597 21661 raft_consensus.cc:2258] T 903409fdfec246f88f037d3420c1a942 P 21a714af281940b497ab06ba85aaa27d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 15:02:07.771860 21662 ts_tablet_manager.cc:1905] T 903409fdfec246f88f037d3420c1a942 P b3dff0a518044ddeb2e65d3fbfb0ed68: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 15:02:07.772033 21660 ts_tablet_manager.cc:1905] T 903409fdfec246f88f037d3420c1a942 P f2f18bf473b04a23a122a756e8f646a3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 15:02:07.772037 21661 ts_tablet_manager.cc:1905] T 903409fdfec246f88f037d3420c1a942 P 21a714af281940b497ab06ba85aaa27d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 15:02:07.774075 21660 ts_tablet_manager.cc:1918] T 903409fdfec246f88f037d3420c1a942 P f2f18bf473b04a23a122a756e8f646a3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260923 15:02:07.774138 21660 log.cc:1196] T 903409fdfec246f88f037d3420c1a942 P f2f18bf473b04a23a122a756e8f646a3: Deleting WAL directory at /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175691491353-13614-0/minicluster-data/ts-0/wal/wals/903409fdfec246f88f037d3420c1a942
I20260923 15:02:07.774355 21660 ts_tablet_manager.cc:1939] T 903409fdfec246f88f037d3420c1a942 P f2f18bf473b04a23a122a756e8f646a3: Deleting consensus metadata
I20260923 15:02:07.774538 21661 ts_tablet_manager.cc:1918] T 903409fdfec246f88f037d3420c1a942 P 21a714af281940b497ab06ba85aaa27d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260923 15:02:07.774605 21661 log.cc:1196] T 903409fdfec246f88f037d3420c1a942 P 21a714af281940b497ab06ba85aaa27d: Deleting WAL directory at /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175691491353-13614-0/minicluster-data/ts-1/wal/wals/903409fdfec246f88f037d3420c1a942
I20260923 15:02:07.774755 21112 catalog_manager.cc:4841] TS f2f18bf473b04a23a122a756e8f646a3 (127.13.75.129:38767): tablet 903409fdfec246f88f037d3420c1a942 (table test-table [id=c68093850b9b42aea4a70ae4a0f355c0]) successfully deleted
I20260923 15:02:07.774837 21661 ts_tablet_manager.cc:1939] T 903409fdfec246f88f037d3420c1a942 P 21a714af281940b497ab06ba85aaa27d: Deleting consensus metadata
I20260923 15:02:07.775117 21662 ts_tablet_manager.cc:1918] T 903409fdfec246f88f037d3420c1a942 P b3dff0a518044ddeb2e65d3fbfb0ed68: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260923 15:02:07.775185 21662 log.cc:1196] T 903409fdfec246f88f037d3420c1a942 P b3dff0a518044ddeb2e65d3fbfb0ed68: Deleting WAL directory at /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175691491353-13614-0/minicluster-data/ts-2/wal/wals/903409fdfec246f88f037d3420c1a942
I20260923 15:02:07.775489 21110 catalog_manager.cc:4841] TS 21a714af281940b497ab06ba85aaa27d (127.13.75.130:46875): tablet 903409fdfec246f88f037d3420c1a942 (table test-table [id=c68093850b9b42aea4a70ae4a0f355c0]) successfully deleted
I20260923 15:02:07.775574 21662 ts_tablet_manager.cc:1939] T 903409fdfec246f88f037d3420c1a942 P b3dff0a518044ddeb2e65d3fbfb0ed68: Deleting consensus metadata
I20260923 15:02:07.776011 21113 catalog_manager.cc:4841] TS b3dff0a518044ddeb2e65d3fbfb0ed68 (127.13.75.131:39525): tablet 903409fdfec246f88f037d3420c1a942 (table test-table [id=c68093850b9b42aea4a70ae4a0f355c0]) successfully deleted
Sep 23 15:02:07 dist-test-slave-cnb0 krb5kdc[21081](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175727, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
Sep 23 15:02:07 dist-test-slave-cnb0 krb5kdc[21081](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.13.75.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 15:02:07 dist-test-slave-cnb0 krb5kdc[21081](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, test-admin@KRBTEST.COM for kudu/127.13.75.190@KRBTEST.COM, Server not found in Kerberos database
W20260923 15:02:07.802911 21175 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 15:02:07.795100 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35154 (local address 127.13.75.190:40515)
0923 15:02:07.795156 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0923 15:02:07.795158 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:02:07.795303 (+ 145us) server_negotiation.cc:407] Connection header received
0923 15:02:07.795345 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:02:07.795346 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:02:07.795358 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:02:07.795451 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:02:07.796299 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:07.797123 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:07.798020 (+ 897us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:07.798338 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:07.798430 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:07.802784 (+ 4354us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:35154: BlockingRecv error: recv got EOF from 127.0.0.1:35154 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
Sep 23 15:02:07 dist-test-slave-cnb0 krb5kdc[21081](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175727, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for oryx/127.13.75.190@KRBTEST.COM
I20260923 15:02:07.817551 21175 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:07.804153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35168 (local address 127.13.75.190:40515)
0923 15:02:07.804230 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0923 15:02:07.804232 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:02:07.807279 (+ 3047us) server_negotiation.cc:407] Connection header received
0923 15:02:07.807316 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:02:07.807317 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:02:07.807330 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:02:07.807437 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:02:07.808210 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:07.808877 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:07.810239 (+ 1362us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:07.810509 (+ 270us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:07.810614 (+ 105us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:07.813998 (+ 3384us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:02:07.814000 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:02:07.814000 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:02:07.814011 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:02:07.815924 (+ 1913us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:02:07.816370 (+ 446us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:02:07.816372 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:02:07.816373 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:02:07.816473 (+ 100us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:02:07.816778 (+ 305us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:02:07.816780 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:02:07.816780 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:02:07.817020 (+ 240us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:02:07.817138 (+ 118us) server_negotiation.cc:1035] Waiting for connection context
0923 15:02:07.817341 (+ 203us) server_negotiation.cc:299] Negotiation successful
0923 15:02:07.817445 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260923 15:02:07.820279 21126 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35168:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 15:02:07.820549 21126 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 15:02:07.822248 21403 tablet_service.cc:1467] Processing CreateTablet for tablet aff7903665f5459298c150b5a9599b8c (DEFAULT_TABLE table=test-table [id=adb8f639111544f582b539d8ea530f06]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 15:02:07.822233 21263 tablet_service.cc:1467] Processing CreateTablet for tablet aff7903665f5459298c150b5a9599b8c (DEFAULT_TABLE table=test-table [id=adb8f639111544f582b539d8ea530f06]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 15:02:07.822311 21543 tablet_service.cc:1467] Processing CreateTablet for tablet aff7903665f5459298c150b5a9599b8c (DEFAULT_TABLE table=test-table [id=adb8f639111544f582b539d8ea530f06]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 15:02:07.822409 21403 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aff7903665f5459298c150b5a9599b8c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:02:07.822423 21543 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aff7903665f5459298c150b5a9599b8c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:02:07.822422 21263 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aff7903665f5459298c150b5a9599b8c. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:02:07.823972 21641 tablet_bootstrap.cc:492] T aff7903665f5459298c150b5a9599b8c P f2f18bf473b04a23a122a756e8f646a3: Bootstrap starting.
I20260923 15:02:07.824172 21638 tablet_bootstrap.cc:492] T aff7903665f5459298c150b5a9599b8c P 21a714af281940b497ab06ba85aaa27d: Bootstrap starting.
I20260923 15:02:07.824504 21641 tablet_bootstrap.cc:654] T aff7903665f5459298c150b5a9599b8c P f2f18bf473b04a23a122a756e8f646a3: Neither blocks nor log segments found. Creating new log.
I20260923 15:02:07.824543 21638 tablet_bootstrap.cc:654] T aff7903665f5459298c150b5a9599b8c P 21a714af281940b497ab06ba85aaa27d: Neither blocks nor log segments found. Creating new log.
I20260923 15:02:07.825484 21641 tablet_bootstrap.cc:492] T aff7903665f5459298c150b5a9599b8c P f2f18bf473b04a23a122a756e8f646a3: No bootstrap required, opened a new log
I20260923 15:02:07.825554 21641 ts_tablet_manager.cc:1397] T aff7903665f5459298c150b5a9599b8c P f2f18bf473b04a23a122a756e8f646a3: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 15:02:07.825724 21641 raft_consensus.cc:348] T aff7903665f5459298c150b5a9599b8c P f2f18bf473b04a23a122a756e8f646a3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2f18bf473b04a23a122a756e8f646a3" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 38767 } } peers { permanent_uuid: "21a714af281940b497ab06ba85aaa27d" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 46875 } } peers { permanent_uuid: "b3dff0a518044ddeb2e65d3fbfb0ed68" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 39525 } }
I20260923 15:02:07.825968 21641 raft_consensus.cc:374] T aff7903665f5459298c150b5a9599b8c P f2f18bf473b04a23a122a756e8f646a3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:02:07.826005 21641 raft_consensus.cc:729] T aff7903665f5459298c150b5a9599b8c P f2f18bf473b04a23a122a756e8f646a3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2f18bf473b04a23a122a756e8f646a3, State: Initialized, Role: FOLLOWER
I20260923 15:02:07.826066 21641 consensus_queue.cc:260] T aff7903665f5459298c150b5a9599b8c P f2f18bf473b04a23a122a756e8f646a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2f18bf473b04a23a122a756e8f646a3" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 38767 } } peers { permanent_uuid: "21a714af281940b497ab06ba85aaa27d" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 46875 } } peers { permanent_uuid: "b3dff0a518044ddeb2e65d3fbfb0ed68" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 39525 } }
I20260923 15:02:07.826169 21641 ts_tablet_manager.cc:1428] T aff7903665f5459298c150b5a9599b8c P f2f18bf473b04a23a122a756e8f646a3: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260923 15:02:07.827025 21637 tablet_bootstrap.cc:492] T aff7903665f5459298c150b5a9599b8c P b3dff0a518044ddeb2e65d3fbfb0ed68: Bootstrap starting.
I20260923 15:02:07.827512 21637 tablet_bootstrap.cc:654] T aff7903665f5459298c150b5a9599b8c P b3dff0a518044ddeb2e65d3fbfb0ed68: Neither blocks nor log segments found. Creating new log.
I20260923 15:02:07.828504 21638 tablet_bootstrap.cc:492] T aff7903665f5459298c150b5a9599b8c P 21a714af281940b497ab06ba85aaa27d: No bootstrap required, opened a new log
I20260923 15:02:07.828531 21637 tablet_bootstrap.cc:492] T aff7903665f5459298c150b5a9599b8c P b3dff0a518044ddeb2e65d3fbfb0ed68: No bootstrap required, opened a new log
I20260923 15:02:07.828572 21638 ts_tablet_manager.cc:1397] T aff7903665f5459298c150b5a9599b8c P 21a714af281940b497ab06ba85aaa27d: Time spent bootstrapping tablet: real 0.004s user 0.000s sys 0.000s
I20260923 15:02:07.828603 21637 ts_tablet_manager.cc:1397] T aff7903665f5459298c150b5a9599b8c P b3dff0a518044ddeb2e65d3fbfb0ed68: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.001s
I20260923 15:02:07.828697 21638 raft_consensus.cc:348] T aff7903665f5459298c150b5a9599b8c P 21a714af281940b497ab06ba85aaa27d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2f18bf473b04a23a122a756e8f646a3" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 38767 } } peers { permanent_uuid: "21a714af281940b497ab06ba85aaa27d" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 46875 } } peers { permanent_uuid: "b3dff0a518044ddeb2e65d3fbfb0ed68" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 39525 } }
I20260923 15:02:07.828786 21638 raft_consensus.cc:374] T aff7903665f5459298c150b5a9599b8c P 21a714af281940b497ab06ba85aaa27d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:02:07.828814 21638 raft_consensus.cc:729] T aff7903665f5459298c150b5a9599b8c P 21a714af281940b497ab06ba85aaa27d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 21a714af281940b497ab06ba85aaa27d, State: Initialized, Role: FOLLOWER
I20260923 15:02:07.828804 21637 raft_consensus.cc:348] T aff7903665f5459298c150b5a9599b8c P b3dff0a518044ddeb2e65d3fbfb0ed68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2f18bf473b04a23a122a756e8f646a3" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 38767 } } peers { permanent_uuid: "21a714af281940b497ab06ba85aaa27d" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 46875 } } peers { permanent_uuid: "b3dff0a518044ddeb2e65d3fbfb0ed68" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 39525 } }
I20260923 15:02:07.828866 21638 consensus_queue.cc:260] T aff7903665f5459298c150b5a9599b8c P 21a714af281940b497ab06ba85aaa27d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2f18bf473b04a23a122a756e8f646a3" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 38767 } } peers { permanent_uuid: "21a714af281940b497ab06ba85aaa27d" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 46875 } } peers { permanent_uuid: "b3dff0a518044ddeb2e65d3fbfb0ed68" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 39525 } }
I20260923 15:02:07.828890 21637 raft_consensus.cc:374] T aff7903665f5459298c150b5a9599b8c P b3dff0a518044ddeb2e65d3fbfb0ed68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:02:07.828914 21637 raft_consensus.cc:729] T aff7903665f5459298c150b5a9599b8c P b3dff0a518044ddeb2e65d3fbfb0ed68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3dff0a518044ddeb2e65d3fbfb0ed68, State: Initialized, Role: FOLLOWER
I20260923 15:02:07.828965 21638 ts_tablet_manager.cc:1428] T aff7903665f5459298c150b5a9599b8c P 21a714af281940b497ab06ba85aaa27d: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260923 15:02:07.828966 21637 consensus_queue.cc:260] T aff7903665f5459298c150b5a9599b8c P b3dff0a518044ddeb2e65d3fbfb0ed68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2f18bf473b04a23a122a756e8f646a3" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 38767 } } peers { permanent_uuid: "21a714af281940b497ab06ba85aaa27d" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 46875 } } peers { permanent_uuid: "b3dff0a518044ddeb2e65d3fbfb0ed68" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 39525 } }
I20260923 15:02:07.829052 21637 ts_tablet_manager.cc:1428] T aff7903665f5459298c150b5a9599b8c P b3dff0a518044ddeb2e65d3fbfb0ed68: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260923 15:02:08.001832 21645 raft_consensus.cc:482] T aff7903665f5459298c150b5a9599b8c P f2f18bf473b04a23a122a756e8f646a3 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 15:02:08.001945 21645 raft_consensus.cc:504] T aff7903665f5459298c150b5a9599b8c P f2f18bf473b04a23a122a756e8f646a3 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2f18bf473b04a23a122a756e8f646a3" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 38767 } } peers { permanent_uuid: "21a714af281940b497ab06ba85aaa27d" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 46875 } } peers { permanent_uuid: "b3dff0a518044ddeb2e65d3fbfb0ed68" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 39525 } }
I20260923 15:02:08.002413 21645 leader_election.cc:288] T aff7903665f5459298c150b5a9599b8c P f2f18bf473b04a23a122a756e8f646a3 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 21a714af281940b497ab06ba85aaa27d (127.13.75.130:46875), b3dff0a518044ddeb2e65d3fbfb0ed68 (127.13.75.131:39525)
I20260923 15:02:08.005774 21707 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:08.002665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.131:39525 (local address 127.13.75.129:33653)
0923 15:02:08.002933 (+ 268us) negotiation.cc:107] Waiting for socket to connect
0923 15:02:08.002944 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 15:02:08.003031 (+ 87us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:02:08.003263 (+ 232us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:02:08.003264 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:02:08.003266 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:02:08.003359 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:08.003361 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:08.004012 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:08.004014 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:08.004855 (+ 841us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:08.004857 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:08.005474 (+ 617us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:08.005476 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:08.005548 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:08.005553 (+ 5us) client_negotiation.cc:769] Sending connection context
0923 15:02:08.005618 (+ 65us) client_negotiation.cc:240] Negotiation successful
0923 15:02:08.005664 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":253,"mutex_wait_us":3,"thread_start_us":101,"threads_started":1}
I20260923 15:02:08.005754 21634 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:08.002789 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.129:33653 (local address 127.13.75.131:39525)
0923 15:02:08.002852 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0923 15:02:08.002854 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:02:08.003017 (+ 163us) server_negotiation.cc:407] Connection header received
0923 15:02:08.003178 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:02:08.003179 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:02:08.003192 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:02:08.003259 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:02:08.003436 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:08.003917 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:08.004968 (+ 1051us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:08.005336 (+ 368us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:08.005445 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:08.005455 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0923 15:02:08.005638 (+ 183us) server_negotiation.cc:299] Negotiation successful
0923 15:02:08.005685 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260923 15:02:08.006058 21563 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aff7903665f5459298c150b5a9599b8c" candidate_uuid: "f2f18bf473b04a23a122a756e8f646a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3dff0a518044ddeb2e65d3fbfb0ed68" is_pre_election: true
I20260923 15:02:08.006182 21563 raft_consensus.cc:2454] T aff7903665f5459298c150b5a9599b8c P b3dff0a518044ddeb2e65d3fbfb0ed68 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2f18bf473b04a23a122a756e8f646a3 in term 0.
I20260923 15:02:08.006390 21217 leader_election.cc:302] T aff7903665f5459298c150b5a9599b8c P f2f18bf473b04a23a122a756e8f646a3 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: b3dff0a518044ddeb2e65d3fbfb0ed68, f2f18bf473b04a23a122a756e8f646a3; no voters:
I20260923 15:02:08.006494 21635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:08.002813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.129:39255 (local address 127.13.75.130:46875)
0923 15:02:08.002907 (+ 94us) server_negotiation.cc:206] Beginning negotiation
0923 15:02:08.002924 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0923 15:02:08.002962 (+ 38us) server_negotiation.cc:407] Connection header received
0923 15:02:08.003343 (+ 381us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:02:08.003344 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:02:08.003360 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:02:08.003416 (+ 56us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:02:08.003623 (+ 207us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:08.004172 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:08.005594 (+ 1422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:08.006095 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:08.006175 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:08.006185 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0923 15:02:08.006374 (+ 189us) server_negotiation.cc:299] Negotiation successful
0923 15:02:08.006428 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":32}
I20260923 15:02:08.006484 21708 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:08.002663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.130:46875 (local address 127.13.75.129:39255)
0923 15:02:08.002854 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0923 15:02:08.002870 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0923 15:02:08.002988 (+ 118us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:02:08.003460 (+ 472us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:02:08.003461 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:02:08.003464 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:02:08.003545 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:08.003546 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:08.004271 (+ 725us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:08.004272 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:08.005464 (+ 1192us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:08.005467 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:08.006215 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:08.006216 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:08.006297 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:08.006302 (+ 5us) client_negotiation.cc:769] Sending connection context
0923 15:02:08.006357 (+ 55us) client_negotiation.cc:240] Negotiation successful
0923 15:02:08.006402 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":74,"threads_started":1}
I20260923 15:02:08.006559 21645 raft_consensus.cc:2781] T aff7903665f5459298c150b5a9599b8c P f2f18bf473b04a23a122a756e8f646a3 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 15:02:08.006608 21645 raft_consensus.cc:482] T aff7903665f5459298c150b5a9599b8c P f2f18bf473b04a23a122a756e8f646a3 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 15:02:08.006629 21645 raft_consensus.cc:3037] T aff7903665f5459298c150b5a9599b8c P f2f18bf473b04a23a122a756e8f646a3 [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:02:08.006914 21423 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aff7903665f5459298c150b5a9599b8c" candidate_uuid: "f2f18bf473b04a23a122a756e8f646a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21a714af281940b497ab06ba85aaa27d" is_pre_election: true
I20260923 15:02:08.007033 21423 raft_consensus.cc:2454] T aff7903665f5459298c150b5a9599b8c P 21a714af281940b497ab06ba85aaa27d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f2f18bf473b04a23a122a756e8f646a3 in term 0.
I20260923 15:02:08.007532 21645 raft_consensus.cc:504] T aff7903665f5459298c150b5a9599b8c P f2f18bf473b04a23a122a756e8f646a3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2f18bf473b04a23a122a756e8f646a3" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 38767 } } peers { permanent_uuid: "21a714af281940b497ab06ba85aaa27d" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 46875 } } peers { permanent_uuid: "b3dff0a518044ddeb2e65d3fbfb0ed68" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 39525 } }
I20260923 15:02:08.007685 21645 leader_election.cc:288] T aff7903665f5459298c150b5a9599b8c P f2f18bf473b04a23a122a756e8f646a3 [CANDIDATE]: Term 1 election: Requested vote from peers 21a714af281940b497ab06ba85aaa27d (127.13.75.130:46875), b3dff0a518044ddeb2e65d3fbfb0ed68 (127.13.75.131:39525)
I20260923 15:02:08.007845 21423 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aff7903665f5459298c150b5a9599b8c" candidate_uuid: "f2f18bf473b04a23a122a756e8f646a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "21a714af281940b497ab06ba85aaa27d"
I20260923 15:02:08.007869 21563 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aff7903665f5459298c150b5a9599b8c" candidate_uuid: "f2f18bf473b04a23a122a756e8f646a3" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3dff0a518044ddeb2e65d3fbfb0ed68"
I20260923 15:02:08.007926 21423 raft_consensus.cc:3037] T aff7903665f5459298c150b5a9599b8c P 21a714af281940b497ab06ba85aaa27d [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:02:08.007951 21563 raft_consensus.cc:3037] T aff7903665f5459298c150b5a9599b8c P b3dff0a518044ddeb2e65d3fbfb0ed68 [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:02:08.008642 21423 raft_consensus.cc:2454] T aff7903665f5459298c150b5a9599b8c P 21a714af281940b497ab06ba85aaa27d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2f18bf473b04a23a122a756e8f646a3 in term 1.
I20260923 15:02:08.008819 21214 leader_election.cc:302] T aff7903665f5459298c150b5a9599b8c P f2f18bf473b04a23a122a756e8f646a3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 21a714af281940b497ab06ba85aaa27d, f2f18bf473b04a23a122a756e8f646a3; no voters:
I20260923 15:02:08.008836 21563 raft_consensus.cc:2454] T aff7903665f5459298c150b5a9599b8c P b3dff0a518044ddeb2e65d3fbfb0ed68 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f2f18bf473b04a23a122a756e8f646a3 in term 1.
I20260923 15:02:08.009001 21645 raft_consensus.cc:2781] T aff7903665f5459298c150b5a9599b8c P f2f18bf473b04a23a122a756e8f646a3 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 15:02:08.009168 21645 raft_consensus.cc:686] T aff7903665f5459298c150b5a9599b8c P f2f18bf473b04a23a122a756e8f646a3 [term 1 LEADER]: Becoming Leader. State: Replica: f2f18bf473b04a23a122a756e8f646a3, State: Running, Role: LEADER
I20260923 15:02:08.009267 21645 consensus_queue.cc:237] T aff7903665f5459298c150b5a9599b8c P f2f18bf473b04a23a122a756e8f646a3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2f18bf473b04a23a122a756e8f646a3" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 38767 } } peers { permanent_uuid: "21a714af281940b497ab06ba85aaa27d" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 46875 } } peers { permanent_uuid: "b3dff0a518044ddeb2e65d3fbfb0ed68" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 39525 } }
I20260923 15:02:08.010042 21125 catalog_manager.cc:5495] T aff7903665f5459298c150b5a9599b8c P f2f18bf473b04a23a122a756e8f646a3 reported cstate change: term changed from 0 to 1, leader changed from <none> to f2f18bf473b04a23a122a756e8f646a3 (127.13.75.129). New cstate: current_term: 1 leader_uuid: "f2f18bf473b04a23a122a756e8f646a3" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2f18bf473b04a23a122a756e8f646a3" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 38767 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "21a714af281940b497ab06ba85aaa27d" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 46875 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "b3dff0a518044ddeb2e65d3fbfb0ed68" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 39525 } health_report { overall_health: UNKNOWN } } }
I20260923 15:02:08.018090 21636 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:08.015552 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:35812 (local address 127.13.75.129:38767)
0923 15:02:08.015623 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0923 15:02:08.015625 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:02:08.015636 (+ 11us) server_negotiation.cc:407] Connection header received
0923 15:02:08.015678 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:02:08.015679 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:02:08.015706 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:02:08.015790 (+ 84us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 15:02:08.015905 (+ 115us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:08.016414 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:08.017176 (+ 762us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:08.017475 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:08.017543 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:08.017733 (+ 190us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 15:02:08.017851 (+ 118us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 15:02:08.017947 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0923 15:02:08.017973 (+ 26us) server_negotiation.cc:299] Negotiation successful
0923 15:02:08.017994 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260923 15:02:08.019347 21423 raft_consensus.cc:1261] T aff7903665f5459298c150b5a9599b8c P 21a714af281940b497ab06ba85aaa27d [term 1 FOLLOWER]: Refusing update from remote peer f2f18bf473b04a23a122a756e8f646a3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 15:02:08.019667 21563 raft_consensus.cc:1261] T aff7903665f5459298c150b5a9599b8c P b3dff0a518044ddeb2e65d3fbfb0ed68 [term 1 FOLLOWER]: Refusing update from remote peer f2f18bf473b04a23a122a756e8f646a3: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 15:02:08.019797 21645 consensus_queue.cc:1035] T aff7903665f5459298c150b5a9599b8c P f2f18bf473b04a23a122a756e8f646a3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "21a714af281940b497ab06ba85aaa27d" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 46875 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 15:02:08.019965 21709 consensus_queue.cc:1035] T aff7903665f5459298c150b5a9599b8c P f2f18bf473b04a23a122a756e8f646a3 [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3dff0a518044ddeb2e65d3fbfb0ed68" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 39525 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 15:02:08.024112 21124 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35168:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 15:02:08.024235 21124 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35168:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 15:02:08.025120 21124 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 385ef6dbf6774aa09aff37a6dcf6afd0: Sending DeleteTablet for 3 replicas of tablet aff7903665f5459298c150b5a9599b8c
I20260923 15:02:08.025429 21403 tablet_service.cc:1514] Processing DeleteTablet for tablet aff7903665f5459298c150b5a9599b8c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 15:02:08 UTC) from {username='oryx', principal='oryx/127.13.75.190@KRBTEST.COM'} at 127.0.0.1:52954
I20260923 15:02:08.025552 21543 tablet_service.cc:1514] Processing DeleteTablet for tablet aff7903665f5459298c150b5a9599b8c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 15:02:08 UTC) from {username='oryx', principal='oryx/127.13.75.190@KRBTEST.COM'} at 127.0.0.1:56734
I20260923 15:02:08.025658 21661 tablet_replica.cc:330] T aff7903665f5459298c150b5a9599b8c P 21a714af281940b497ab06ba85aaa27d: stopping tablet replica
I20260923 15:02:08.025735 21661 raft_consensus.cc:2229] T aff7903665f5459298c150b5a9599b8c P 21a714af281940b497ab06ba85aaa27d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 15:02:08.025795 21661 raft_consensus.cc:2258] T aff7903665f5459298c150b5a9599b8c P 21a714af281940b497ab06ba85aaa27d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 15:02:08.025883 21662 tablet_replica.cc:330] T aff7903665f5459298c150b5a9599b8c P b3dff0a518044ddeb2e65d3fbfb0ed68: stopping tablet replica
I20260923 15:02:08.025839 21124 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35168:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
I20260923 15:02:08.025949 21662 raft_consensus.cc:2229] T aff7903665f5459298c150b5a9599b8c P b3dff0a518044ddeb2e65d3fbfb0ed68 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 15:02:08.025996 21662 raft_consensus.cc:2258] T aff7903665f5459298c150b5a9599b8c P b3dff0a518044ddeb2e65d3fbfb0ed68 [term 1 FOLLOWER]: Raft consensus is shut down!
W20260923 15:02:08.026023 21124 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 15:02:08.026226 21263 tablet_service.cc:1514] Processing DeleteTablet for tablet aff7903665f5459298c150b5a9599b8c with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 15:02:08 UTC) from {username='oryx', principal='oryx/127.13.75.190@KRBTEST.COM'} at 127.0.0.1:35798
I20260923 15:02:08.026319 21660 tablet_replica.cc:330] T aff7903665f5459298c150b5a9599b8c P f2f18bf473b04a23a122a756e8f646a3: stopping tablet replica
I20260923 15:02:08.026366 21660 raft_consensus.cc:2229] T aff7903665f5459298c150b5a9599b8c P f2f18bf473b04a23a122a756e8f646a3 [term 1 LEADER]: Raft consensus shutting down.
I20260923 15:02:08.026463 21660 raft_consensus.cc:2258] T aff7903665f5459298c150b5a9599b8c P f2f18bf473b04a23a122a756e8f646a3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 15:02:08.026614 21662 ts_tablet_manager.cc:1905] T aff7903665f5459298c150b5a9599b8c P b3dff0a518044ddeb2e65d3fbfb0ed68: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 15:02:08.026643 21661 ts_tablet_manager.cc:1905] T aff7903665f5459298c150b5a9599b8c P 21a714af281940b497ab06ba85aaa27d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 15:02:08.026968 21660 ts_tablet_manager.cc:1905] T aff7903665f5459298c150b5a9599b8c P f2f18bf473b04a23a122a756e8f646a3: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 15:02:08.028637 21661 ts_tablet_manager.cc:1918] T aff7903665f5459298c150b5a9599b8c P 21a714af281940b497ab06ba85aaa27d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 15:02:08.028702 21661 log.cc:1196] T aff7903665f5459298c150b5a9599b8c P 21a714af281940b497ab06ba85aaa27d: Deleting WAL directory at /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175691491353-13614-0/minicluster-data/ts-1/wal/wals/aff7903665f5459298c150b5a9599b8c
I20260923 15:02:08.028752 21662 ts_tablet_manager.cc:1918] T aff7903665f5459298c150b5a9599b8c P b3dff0a518044ddeb2e65d3fbfb0ed68: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 15:02:08.028811 21662 log.cc:1196] T aff7903665f5459298c150b5a9599b8c P b3dff0a518044ddeb2e65d3fbfb0ed68: Deleting WAL directory at /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175691491353-13614-0/minicluster-data/ts-2/wal/wals/aff7903665f5459298c150b5a9599b8c
I20260923 15:02:08.028919 21661 ts_tablet_manager.cc:1939] T aff7903665f5459298c150b5a9599b8c P 21a714af281940b497ab06ba85aaa27d: Deleting consensus metadata
I20260923 15:02:08.029068 21662 ts_tablet_manager.cc:1939] T aff7903665f5459298c150b5a9599b8c P b3dff0a518044ddeb2e65d3fbfb0ed68: Deleting consensus metadata
I20260923 15:02:08.029312 21110 catalog_manager.cc:4841] TS 21a714af281940b497ab06ba85aaa27d (127.13.75.130:46875): tablet aff7903665f5459298c150b5a9599b8c (table test-table [id=adb8f639111544f582b539d8ea530f06]) successfully deleted
I20260923 15:02:08.029464 21113 catalog_manager.cc:4841] TS b3dff0a518044ddeb2e65d3fbfb0ed68 (127.13.75.131:39525): tablet aff7903665f5459298c150b5a9599b8c (table test-table [id=adb8f639111544f582b539d8ea530f06]) successfully deleted
I20260923 15:02:08.029770 21543 tablet_service.cc:1467] Processing CreateTablet for tablet 92611668a7424f2fb284317bdf03c108 (DEFAULT_TABLE table=test-table [id=fd8ae0fcd92b4d19b9e0ee10bbf38b6f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 15:02:08.029892 21543 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 92611668a7424f2fb284317bdf03c108. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:02:08.030606 21660 ts_tablet_manager.cc:1918] T aff7903665f5459298c150b5a9599b8c P f2f18bf473b04a23a122a756e8f646a3: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 15:02:08.030710 21660 log.cc:1196] T aff7903665f5459298c150b5a9599b8c P f2f18bf473b04a23a122a756e8f646a3: Deleting WAL directory at /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175691491353-13614-0/minicluster-data/ts-0/wal/wals/aff7903665f5459298c150b5a9599b8c
I20260923 15:02:08.030967 21660 ts_tablet_manager.cc:1939] T aff7903665f5459298c150b5a9599b8c P f2f18bf473b04a23a122a756e8f646a3: Deleting consensus metadata
I20260923 15:02:08.031263 21403 tablet_service.cc:1467] Processing CreateTablet for tablet 92611668a7424f2fb284317bdf03c108 (DEFAULT_TABLE table=test-table [id=fd8ae0fcd92b4d19b9e0ee10bbf38b6f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 15:02:08.031400 21403 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 92611668a7424f2fb284317bdf03c108. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:02:08.032725 21638 tablet_bootstrap.cc:492] T 92611668a7424f2fb284317bdf03c108 P 21a714af281940b497ab06ba85aaa27d: Bootstrap starting.
I20260923 15:02:08.032730 21637 tablet_bootstrap.cc:492] T 92611668a7424f2fb284317bdf03c108 P b3dff0a518044ddeb2e65d3fbfb0ed68: Bootstrap starting.
I20260923 15:02:08.033131 21638 tablet_bootstrap.cc:654] T 92611668a7424f2fb284317bdf03c108 P 21a714af281940b497ab06ba85aaa27d: Neither blocks nor log segments found. Creating new log.
I20260923 15:02:08.033164 21637 tablet_bootstrap.cc:654] T 92611668a7424f2fb284317bdf03c108 P b3dff0a518044ddeb2e65d3fbfb0ed68: Neither blocks nor log segments found. Creating new log.
I20260923 15:02:08.033463 21263 tablet_service.cc:1467] Processing CreateTablet for tablet 92611668a7424f2fb284317bdf03c108 (DEFAULT_TABLE table=test-table [id=fd8ae0fcd92b4d19b9e0ee10bbf38b6f]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 15:02:08.033586 21263 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 92611668a7424f2fb284317bdf03c108. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:02:08.033934 21637 tablet_bootstrap.cc:492] T 92611668a7424f2fb284317bdf03c108 P b3dff0a518044ddeb2e65d3fbfb0ed68: No bootstrap required, opened a new log
I20260923 15:02:08.034026 21637 ts_tablet_manager.cc:1397] T 92611668a7424f2fb284317bdf03c108 P b3dff0a518044ddeb2e65d3fbfb0ed68: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260923 15:02:08.034176 21637 raft_consensus.cc:348] T 92611668a7424f2fb284317bdf03c108 P b3dff0a518044ddeb2e65d3fbfb0ed68 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2f18bf473b04a23a122a756e8f646a3" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 38767 } } peers { permanent_uuid: "21a714af281940b497ab06ba85aaa27d" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 46875 } } peers { permanent_uuid: "b3dff0a518044ddeb2e65d3fbfb0ed68" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 39525 } }
I20260923 15:02:08.034277 21637 raft_consensus.cc:374] T 92611668a7424f2fb284317bdf03c108 P b3dff0a518044ddeb2e65d3fbfb0ed68 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:02:08.034322 21637 raft_consensus.cc:729] T 92611668a7424f2fb284317bdf03c108 P b3dff0a518044ddeb2e65d3fbfb0ed68 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b3dff0a518044ddeb2e65d3fbfb0ed68, State: Initialized, Role: FOLLOWER
I20260923 15:02:08.034370 21637 consensus_queue.cc:260] T 92611668a7424f2fb284317bdf03c108 P b3dff0a518044ddeb2e65d3fbfb0ed68 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2f18bf473b04a23a122a756e8f646a3" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 38767 } } peers { permanent_uuid: "21a714af281940b497ab06ba85aaa27d" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 46875 } } peers { permanent_uuid: "b3dff0a518044ddeb2e65d3fbfb0ed68" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 39525 } }
I20260923 15:02:08.034482 21637 ts_tablet_manager.cc:1428] T 92611668a7424f2fb284317bdf03c108 P b3dff0a518044ddeb2e65d3fbfb0ed68: Time spent starting tablet: real 0.000s user 0.000s sys 0.001s
I20260923 15:02:08.034902 21641 tablet_bootstrap.cc:492] T 92611668a7424f2fb284317bdf03c108 P f2f18bf473b04a23a122a756e8f646a3: Bootstrap starting.
I20260923 15:02:08.034983 21112 catalog_manager.cc:4841] TS f2f18bf473b04a23a122a756e8f646a3 (127.13.75.129:38767): tablet aff7903665f5459298c150b5a9599b8c (table test-table [id=adb8f639111544f582b539d8ea530f06]) successfully deleted
I20260923 15:02:08.035358 21641 tablet_bootstrap.cc:654] T 92611668a7424f2fb284317bdf03c108 P f2f18bf473b04a23a122a756e8f646a3: Neither blocks nor log segments found. Creating new log.
I20260923 15:02:08.035844 21638 tablet_bootstrap.cc:492] T 92611668a7424f2fb284317bdf03c108 P 21a714af281940b497ab06ba85aaa27d: No bootstrap required, opened a new log
I20260923 15:02:08.035913 21638 ts_tablet_manager.cc:1397] T 92611668a7424f2fb284317bdf03c108 P 21a714af281940b497ab06ba85aaa27d: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.000s
I20260923 15:02:08.036062 21641 tablet_bootstrap.cc:492] T 92611668a7424f2fb284317bdf03c108 P f2f18bf473b04a23a122a756e8f646a3: No bootstrap required, opened a new log
I20260923 15:02:08.036116 21641 ts_tablet_manager.cc:1397] T 92611668a7424f2fb284317bdf03c108 P f2f18bf473b04a23a122a756e8f646a3: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260923 15:02:08.036042 21638 raft_consensus.cc:348] T 92611668a7424f2fb284317bdf03c108 P 21a714af281940b497ab06ba85aaa27d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2f18bf473b04a23a122a756e8f646a3" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 38767 } } peers { permanent_uuid: "21a714af281940b497ab06ba85aaa27d" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 46875 } } peers { permanent_uuid: "b3dff0a518044ddeb2e65d3fbfb0ed68" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 39525 } }
I20260923 15:02:08.036144 21638 raft_consensus.cc:374] T 92611668a7424f2fb284317bdf03c108 P 21a714af281940b497ab06ba85aaa27d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:02:08.036183 21638 raft_consensus.cc:729] T 92611668a7424f2fb284317bdf03c108 P 21a714af281940b497ab06ba85aaa27d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 21a714af281940b497ab06ba85aaa27d, State: Initialized, Role: FOLLOWER
I20260923 15:02:08.036244 21638 consensus_queue.cc:260] T 92611668a7424f2fb284317bdf03c108 P 21a714af281940b497ab06ba85aaa27d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2f18bf473b04a23a122a756e8f646a3" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 38767 } } peers { permanent_uuid: "21a714af281940b497ab06ba85aaa27d" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 46875 } } peers { permanent_uuid: "b3dff0a518044ddeb2e65d3fbfb0ed68" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 39525 } }
I20260923 15:02:08.036286 21641 raft_consensus.cc:348] T 92611668a7424f2fb284317bdf03c108 P f2f18bf473b04a23a122a756e8f646a3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2f18bf473b04a23a122a756e8f646a3" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 38767 } } peers { permanent_uuid: "21a714af281940b497ab06ba85aaa27d" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 46875 } } peers { permanent_uuid: "b3dff0a518044ddeb2e65d3fbfb0ed68" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 39525 } }
I20260923 15:02:08.036369 21638 ts_tablet_manager.cc:1428] T 92611668a7424f2fb284317bdf03c108 P 21a714af281940b497ab06ba85aaa27d: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260923 15:02:08.036391 21641 raft_consensus.cc:374] T 92611668a7424f2fb284317bdf03c108 P f2f18bf473b04a23a122a756e8f646a3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:02:08.036415 21641 raft_consensus.cc:729] T 92611668a7424f2fb284317bdf03c108 P f2f18bf473b04a23a122a756e8f646a3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f2f18bf473b04a23a122a756e8f646a3, State: Initialized, Role: FOLLOWER
I20260923 15:02:08.036465 21641 consensus_queue.cc:260] T 92611668a7424f2fb284317bdf03c108 P f2f18bf473b04a23a122a756e8f646a3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2f18bf473b04a23a122a756e8f646a3" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 38767 } } peers { permanent_uuid: "21a714af281940b497ab06ba85aaa27d" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 46875 } } peers { permanent_uuid: "b3dff0a518044ddeb2e65d3fbfb0ed68" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 39525 } }
I20260923 15:02:08.036556 21641 ts_tablet_manager.cc:1428] T 92611668a7424f2fb284317bdf03c108 P f2f18bf473b04a23a122a756e8f646a3: Time spent starting tablet: real 0.000s user 0.001s sys 0.000s
I20260923 15:02:08.234163 21655 raft_consensus.cc:482] T 92611668a7424f2fb284317bdf03c108 P 21a714af281940b497ab06ba85aaa27d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 15:02:08.234277 21655 raft_consensus.cc:504] T 92611668a7424f2fb284317bdf03c108 P 21a714af281940b497ab06ba85aaa27d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2f18bf473b04a23a122a756e8f646a3" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 38767 } } peers { permanent_uuid: "21a714af281940b497ab06ba85aaa27d" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 46875 } } peers { permanent_uuid: "b3dff0a518044ddeb2e65d3fbfb0ed68" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 39525 } }
I20260923 15:02:08.234519 21655 leader_election.cc:288] T 92611668a7424f2fb284317bdf03c108 P 21a714af281940b497ab06ba85aaa27d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f2f18bf473b04a23a122a756e8f646a3 (127.13.75.129:38767), b3dff0a518044ddeb2e65d3fbfb0ed68 (127.13.75.131:39525)
I20260923 15:02:08.234742 21283 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "92611668a7424f2fb284317bdf03c108" candidate_uuid: "21a714af281940b497ab06ba85aaa27d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2f18bf473b04a23a122a756e8f646a3" is_pre_election: true
I20260923 15:02:08.234867 21283 raft_consensus.cc:2454] T 92611668a7424f2fb284317bdf03c108 P f2f18bf473b04a23a122a756e8f646a3 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 21a714af281940b497ab06ba85aaa27d in term 0.
I20260923 15:02:08.234841 21563 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "92611668a7424f2fb284317bdf03c108" candidate_uuid: "21a714af281940b497ab06ba85aaa27d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3dff0a518044ddeb2e65d3fbfb0ed68" is_pre_election: true
I20260923 15:02:08.234988 21563 raft_consensus.cc:2454] T 92611668a7424f2fb284317bdf03c108 P b3dff0a518044ddeb2e65d3fbfb0ed68 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 21a714af281940b497ab06ba85aaa27d in term 0.
I20260923 15:02:08.235154 21356 leader_election.cc:302] T 92611668a7424f2fb284317bdf03c108 P 21a714af281940b497ab06ba85aaa27d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 21a714af281940b497ab06ba85aaa27d, f2f18bf473b04a23a122a756e8f646a3; no voters:
I20260923 15:02:08.235286 21655 raft_consensus.cc:2781] T 92611668a7424f2fb284317bdf03c108 P 21a714af281940b497ab06ba85aaa27d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 15:02:08.235369 21655 raft_consensus.cc:482] T 92611668a7424f2fb284317bdf03c108 P 21a714af281940b497ab06ba85aaa27d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 15:02:08.235404 21655 raft_consensus.cc:3037] T 92611668a7424f2fb284317bdf03c108 P 21a714af281940b497ab06ba85aaa27d [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:02:08.236348 21655 raft_consensus.cc:504] T 92611668a7424f2fb284317bdf03c108 P 21a714af281940b497ab06ba85aaa27d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2f18bf473b04a23a122a756e8f646a3" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 38767 } } peers { permanent_uuid: "21a714af281940b497ab06ba85aaa27d" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 46875 } } peers { permanent_uuid: "b3dff0a518044ddeb2e65d3fbfb0ed68" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 39525 } }
I20260923 15:02:08.236506 21655 leader_election.cc:288] T 92611668a7424f2fb284317bdf03c108 P 21a714af281940b497ab06ba85aaa27d [CANDIDATE]: Term 1 election: Requested vote from peers f2f18bf473b04a23a122a756e8f646a3 (127.13.75.129:38767), b3dff0a518044ddeb2e65d3fbfb0ed68 (127.13.75.131:39525)
I20260923 15:02:08.236733 21283 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "92611668a7424f2fb284317bdf03c108" candidate_uuid: "21a714af281940b497ab06ba85aaa27d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f2f18bf473b04a23a122a756e8f646a3"
I20260923 15:02:08.236721 21563 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "92611668a7424f2fb284317bdf03c108" candidate_uuid: "21a714af281940b497ab06ba85aaa27d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "b3dff0a518044ddeb2e65d3fbfb0ed68"
I20260923 15:02:08.236814 21563 raft_consensus.cc:3037] T 92611668a7424f2fb284317bdf03c108 P b3dff0a518044ddeb2e65d3fbfb0ed68 [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:02:08.236814 21283 raft_consensus.cc:3037] T 92611668a7424f2fb284317bdf03c108 P f2f18bf473b04a23a122a756e8f646a3 [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:02:08.237628 21283 raft_consensus.cc:2454] T 92611668a7424f2fb284317bdf03c108 P f2f18bf473b04a23a122a756e8f646a3 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 21a714af281940b497ab06ba85aaa27d in term 1.
I20260923 15:02:08.237628 21563 raft_consensus.cc:2454] T 92611668a7424f2fb284317bdf03c108 P b3dff0a518044ddeb2e65d3fbfb0ed68 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 21a714af281940b497ab06ba85aaa27d in term 1.
I20260923 15:02:08.237825 21356 leader_election.cc:302] T 92611668a7424f2fb284317bdf03c108 P 21a714af281940b497ab06ba85aaa27d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 21a714af281940b497ab06ba85aaa27d, f2f18bf473b04a23a122a756e8f646a3; no voters:
I20260923 15:02:08.237955 21655 raft_consensus.cc:2781] T 92611668a7424f2fb284317bdf03c108 P 21a714af281940b497ab06ba85aaa27d [term 1 FOLLOWER]: Leader election won for term 1
I20260923 15:02:08.238042 21655 raft_consensus.cc:686] T 92611668a7424f2fb284317bdf03c108 P 21a714af281940b497ab06ba85aaa27d [term 1 LEADER]: Becoming Leader. State: Replica: 21a714af281940b497ab06ba85aaa27d, State: Running, Role: LEADER
I20260923 15:02:08.238102 21655 consensus_queue.cc:237] T 92611668a7424f2fb284317bdf03c108 P 21a714af281940b497ab06ba85aaa27d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2f18bf473b04a23a122a756e8f646a3" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 38767 } } peers { permanent_uuid: "21a714af281940b497ab06ba85aaa27d" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 46875 } } peers { permanent_uuid: "b3dff0a518044ddeb2e65d3fbfb0ed68" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 39525 } }
I20260923 15:02:08.238719 21125 catalog_manager.cc:5495] T 92611668a7424f2fb284317bdf03c108 P 21a714af281940b497ab06ba85aaa27d reported cstate change: term changed from 0 to 1, leader changed from <none> to 21a714af281940b497ab06ba85aaa27d (127.13.75.130). New cstate: current_term: 1 leader_uuid: "21a714af281940b497ab06ba85aaa27d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f2f18bf473b04a23a122a756e8f646a3" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 38767 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "21a714af281940b497ab06ba85aaa27d" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 46875 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "b3dff0a518044ddeb2e65d3fbfb0ed68" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 39525 } health_report { overall_health: UNKNOWN } } }
I20260923 15:02:08.269526 21635 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:08.266665 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52972 (local address 127.13.75.130:46875)
0923 15:02:08.266783 (+ 118us) server_negotiation.cc:206] Beginning negotiation
0923 15:02:08.266785 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:02:08.266797 (+ 12us) server_negotiation.cc:407] Connection header received
0923 15:02:08.266990 (+ 193us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:02:08.266991 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:02:08.267002 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:02:08.267069 (+ 67us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 15:02:08.267202 (+ 133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:08.267730 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:08.268703 (+ 973us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:08.268896 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:08.268968 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:08.269156 (+ 188us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 15:02:08.269183 (+ 27us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 15:02:08.269223 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0923 15:02:08.269358 (+ 135us) server_negotiation.cc:299] Negotiation successful
0923 15:02:08.269377 (+ 19us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":95}
I20260923 15:02:08.272028 21563 raft_consensus.cc:1261] T 92611668a7424f2fb284317bdf03c108 P b3dff0a518044ddeb2e65d3fbfb0ed68 [term 1 FOLLOWER]: Refusing update from remote peer 21a714af281940b497ab06ba85aaa27d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 15:02:08.272029 21283 raft_consensus.cc:1261] T 92611668a7424f2fb284317bdf03c108 P f2f18bf473b04a23a122a756e8f646a3 [term 1 FOLLOWER]: Refusing update from remote peer 21a714af281940b497ab06ba85aaa27d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 15:02:08.272248 21655 consensus_queue.cc:1035] T 92611668a7424f2fb284317bdf03c108 P 21a714af281940b497ab06ba85aaa27d [LEADER]: Connected to new peer: Peer: permanent_uuid: "b3dff0a518044ddeb2e65d3fbfb0ed68" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 39525 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 15:02:08.272466 21647 consensus_queue.cc:1035] T 92611668a7424f2fb284317bdf03c108 P 21a714af281940b497ab06ba85aaa27d [LEADER]: Connected to new peer: Peer: permanent_uuid: "f2f18bf473b04a23a122a756e8f646a3" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 38767 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 15:02:08.274020 21653 mvcc.cc:205] Tried to move back new op lower bound from 7332559783000915968 to 7332559782863691776. Current Snapshot: MvccSnapshot[applied={T|T < 7332559783000915968}]
I20260923 15:02:08.283509 21125 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35168:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 15:02:08.283625 21125 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:35168:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 15:02:08.284390 21125 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 385ef6dbf6774aa09aff37a6dcf6afd0: Sending DeleteTablet for 3 replicas of tablet 92611668a7424f2fb284317bdf03c108
I20260923 15:02:08.284700 21403 tablet_service.cc:1514] Processing DeleteTablet for tablet 92611668a7424f2fb284317bdf03c108 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 15:02:08 UTC) from {username='oryx', principal='oryx/127.13.75.190@KRBTEST.COM'} at 127.0.0.1:52954
I20260923 15:02:08.284700 21543 tablet_service.cc:1514] Processing DeleteTablet for tablet 92611668a7424f2fb284317bdf03c108 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 15:02:08 UTC) from {username='oryx', principal='oryx/127.13.75.190@KRBTEST.COM'} at 127.0.0.1:56734
I20260923 15:02:08.284817 21263 tablet_service.cc:1514] Processing DeleteTablet for tablet 92611668a7424f2fb284317bdf03c108 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 15:02:08 UTC) from {username='oryx', principal='oryx/127.13.75.190@KRBTEST.COM'} at 127.0.0.1:35798
I20260923 15:02:08.284917 21662 tablet_replica.cc:330] T 92611668a7424f2fb284317bdf03c108 P b3dff0a518044ddeb2e65d3fbfb0ed68: stopping tablet replica
I20260923 15:02:08.285013 21662 raft_consensus.cc:2229] T 92611668a7424f2fb284317bdf03c108 P b3dff0a518044ddeb2e65d3fbfb0ed68 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 15:02:08.285076 21662 raft_consensus.cc:2258] T 92611668a7424f2fb284317bdf03c108 P b3dff0a518044ddeb2e65d3fbfb0ed68 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 15:02:08.285106 13614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKC4soy/build/release/bin/kudu with pid 21198
I20260923 15:02:08.285151 21661 tablet_replica.cc:330] T 92611668a7424f2fb284317bdf03c108 P 21a714af281940b497ab06ba85aaa27d: stopping tablet replica
I20260923 15:02:08.285210 21661 raft_consensus.cc:2229] T 92611668a7424f2fb284317bdf03c108 P 21a714af281940b497ab06ba85aaa27d [term 1 LEADER]: Raft consensus shutting down.
I20260923 15:02:08.285287 21661 raft_consensus.cc:2258] T 92611668a7424f2fb284317bdf03c108 P 21a714af281940b497ab06ba85aaa27d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 15:02:08.285362 21660 tablet_replica.cc:330] T 92611668a7424f2fb284317bdf03c108 P f2f18bf473b04a23a122a756e8f646a3: stopping tablet replica
I20260923 15:02:08.285414 21660 raft_consensus.cc:2229] T 92611668a7424f2fb284317bdf03c108 P f2f18bf473b04a23a122a756e8f646a3 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 15:02:08.285449 21660 raft_consensus.cc:2258] T 92611668a7424f2fb284317bdf03c108 P f2f18bf473b04a23a122a756e8f646a3 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 15:02:08.286019 21661 ts_tablet_manager.cc:1905] T 92611668a7424f2fb284317bdf03c108 P 21a714af281940b497ab06ba85aaa27d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 15:02:08.286350 21662 ts_tablet_manager.cc:1905] T 92611668a7424f2fb284317bdf03c108 P b3dff0a518044ddeb2e65d3fbfb0ed68: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 15:02:08.287868 21661 ts_tablet_manager.cc:1918] T 92611668a7424f2fb284317bdf03c108 P 21a714af281940b497ab06ba85aaa27d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260923 15:02:08.287938 21661 log.cc:1196] T 92611668a7424f2fb284317bdf03c108 P 21a714af281940b497ab06ba85aaa27d: Deleting WAL directory at /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175691491353-13614-0/minicluster-data/ts-1/wal/wals/92611668a7424f2fb284317bdf03c108
I20260923 15:02:08.287940 21662 ts_tablet_manager.cc:1918] T 92611668a7424f2fb284317bdf03c108 P b3dff0a518044ddeb2e65d3fbfb0ed68: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.5
I20260923 15:02:08.287987 21662 log.cc:1196] T 92611668a7424f2fb284317bdf03c108 P b3dff0a518044ddeb2e65d3fbfb0ed68: Deleting WAL directory at /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonDefaultPrincipal.1790175691491353-13614-0/minicluster-data/ts-2/wal/wals/92611668a7424f2fb284317bdf03c108
I20260923 15:02:08.288166 21661 ts_tablet_manager.cc:1939] T 92611668a7424f2fb284317bdf03c108 P 21a714af281940b497ab06ba85aaa27d: Deleting consensus metadata
I20260923 15:02:08.288188 21662 ts_tablet_manager.cc:1939] T 92611668a7424f2fb284317bdf03c108 P b3dff0a518044ddeb2e65d3fbfb0ed68: Deleting consensus metadata
I20260923 15:02:08.288870 21113 catalog_manager.cc:4841] TS b3dff0a518044ddeb2e65d3fbfb0ed68 (127.13.75.131:39525): tablet 92611668a7424f2fb284317bdf03c108 (table test-table [id=fd8ae0fcd92b4d19b9e0ee10bbf38b6f]) successfully deleted
I20260923 15:02:08.288941 21110 catalog_manager.cc:4841] TS 21a714af281940b497ab06ba85aaa27d (127.13.75.130:46875): tablet 92611668a7424f2fb284317bdf03c108 (table test-table [id=fd8ae0fcd92b4d19b9e0ee10bbf38b6f]) successfully deleted
W20260923 15:02:08.292965 21112 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.75.129:38767 (error 108)
I20260923 15:02:08.293164 13614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKC4soy/build/release/bin/kudu with pid 21338
I20260923 15:02:08.294509 21726 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:08.294290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.129:38767 (local address 127.0.0.1:35818)
0923 15:02:08.294405 (+ 115us) negotiation.cc:107] Waiting for socket to connect
0923 15:02:08.294468 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.129:38767: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":42,"threads_started":1}
W20260923 15:02:08.294677 21112 catalog_manager.cc:4584] TS f2f18bf473b04a23a122a756e8f646a3 (127.13.75.129:38767): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 92611668a7424f2fb284317bdf03c108: Network error: Client connection negotiation failed: client connection to 127.13.75.129:38767: connect: Connection refused (error 111)
I20260923 15:02:08.299813 13614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKC4soy/build/release/bin/kudu with pid 21478
I20260923 15:02:08.305922 13614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKC4soy/build/release/bin/kudu with pid 21094
2026-09-23T15:02:08Z chronyd exiting
[ OK ] SecurityITest.TestNonDefaultPrincipal (2812 ms)
[ RUN ] SecurityITest.TestNonExistentPrincipal
Loading random data
Initializing database '/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790175691491353-13614-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 15:02:08 dist-test-slave-cnb0 krb5kdc[21729](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 15:02:08 dist-test-slave-cnb0 krb5kdc[21729](info): set up 2 sockets
Sep 23 15:02:08 dist-test-slave-cnb0 krb5kdc[21729](info): commencing operation
krb5kdc: starting...
W20260923 15:02:09.377496 13614 mini_kdc.cc:121] Time spent starting KDC: real 1.050s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790175691491353-13614-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790175691491353-13614-0/krb5kdc/test-user.keytab.
Sep 23 15:02:09 dist-test-slave-cnb0 krb5kdc[21729](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175729, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T15:02:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T15:02:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.75.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.75.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.75.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.190.keytab.
Entry for principal kudu/127.13.75.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.190.keytab.
I20260923 15:02:09.510700 13614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKC4soy/build/release/bin/kudu
/tmp/dist-test-taskKC4soy/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790175691491353-13614-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790175691491353-13614-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790175691491353-13614-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790175691491353-13614-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.75.190:36293
--webserver_interface=127.13.75.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.190.keytab
--principal=kudu/127.13.75.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.75.148:35521
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.75.190:36293
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--principal=oryx with env {KRB5CCNAME=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790175691491353-13614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790175691491353-13614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790175691491353-13614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:02:09.595402 21742 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:02:09.595664 21742 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:02:09.595690 21742 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:02:09.597078 21742 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 15:02:09.597127 21742 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:02:09.597144 21742 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:02:09.597159 21742 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 15:02:09.597173 21742 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 15:02:09.599239 21742 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.75.148:35521
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790175691491353-13614-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790175691491353-13614-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.75.190:36293
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.190.keytab
--principal=oryx
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.75.190:36293
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790175691491353-13614-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.75.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21742
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestNonExistentPrincipal.1790175691491353-13614-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:02:09.599562 21742 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:02:09.599776 21742 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:02:09.605000 21748 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:02:09.605000 21750 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:02:09.605082 21747 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:02:09.605192 21742 server_base.cc:1034] running on GCE node
I20260923 15:02:09.605496 21742 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:02:09.605826 21742 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:02:09.606998 21742 hybrid_clock.cc:648] HybridClock initialized: now 1790175729606973 us; error 44 us; skew 500 ppm
Runtime error: kinit failed: unable to login from keytab: Keytab contains no suitable keys for oryx@KRBTEST.COM
2026-09-23T15:02:09Z chronyd exiting
[ OK ] SecurityITest.TestNonExistentPrincipal (1301 ms)
[ RUN ] SecurityITest.TestMismatchingPrincipals
Loading random data
Initializing database '/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175691491353-13614-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 15:02:09 dist-test-slave-cnb0 krb5kdc[21755](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 15:02:09 dist-test-slave-cnb0 krb5kdc[21755](info): set up 2 sockets
Sep 23 15:02:09 dist-test-slave-cnb0 krb5kdc[21755](info): commencing operation
krb5kdc: starting...
W20260923 15:02:10.713495 13614 mini_kdc.cc:121] Time spent starting KDC: real 1.084s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175691491353-13614-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175691491353-13614-0/krb5kdc/test-user.keytab.
Sep 23 15:02:10 dist-test-slave-cnb0 krb5kdc[21755](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175730, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T15:02:10Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T15:02:10Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.75.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.75.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.75.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.190.keytab.
Entry for principal kudu/127.13.75.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.190.keytab.
I20260923 15:02:10.847911 13614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKC4soy/build/release/bin/kudu
/tmp/dist-test-taskKC4soy/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175691491353-13614-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175691491353-13614-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175691491353-13614-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175691491353-13614-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.75.190:40153
--webserver_interface=127.13.75.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.190.keytab
--principal=kudu/127.13.75.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.75.148:35183
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.75.190:40153
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175691491353-13614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175691491353-13614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175691491353-13614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:02:10.936419 21768 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:02:10.936661 21768 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:02:10.936685 21768 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:02:10.937944 21768 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 15:02:10.937988 21768 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:02:10.938009 21768 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:02:10.938023 21768 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 15:02:10.938035 21768 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 15:02:10.939751 21768 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.75.148:35183
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175691491353-13614-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175691491353-13614-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.75.190:40153
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.190.keytab
--principal=kudu/127.13.75.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.75.190:40153
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175691491353-13614-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.75.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21768
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175691491353-13614-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:02:10.939985 21768 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:02:10.940186 21768 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:02:10.945376 21774 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:02:10.945394 21773 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:02:10.945416 21768 server_base.cc:1034] running on GCE node
W20260923 15:02:10.945396 21776 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:02:10.945921 21768 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:02:10.946256 21768 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:02:10.947414 21768 hybrid_clock.cc:648] HybridClock initialized: now 1790175730947390 us; error 34 us; skew 500 ppm
Sep 23 15:02:10 dist-test-slave-cnb0 krb5kdc[21755](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175730, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:02:10.955113 21768 init.cc:377] Logged in from keytab as kudu/127.13.75.190@KRBTEST.COM (short username kudu)
I20260923 15:02:10.955713 21768 webserver.cc:460] Webserver started at http://127.13.75.190:34373/ using document root <none> and password file <none>
I20260923 15:02:10.956003 21768 fs_manager.cc:359] Metadata directory not provided
I20260923 15:02:10.956084 21768 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:02:10.956252 21768 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:02:10.957548 21768 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175691491353-13614-0/minicluster-data/master-0/data/instance:
uuid: "07e5a7e3b63e4a1fb0c992a164407352"
format_stamp: "Formatted at 2026-09-23 15:02:10 on dist-test-slave-cnb0"
server_key: "9c89b5e11cabfad988408c40a99426e6"
server_key_iv: "bb2440d565f72ce876e51b5a26c3e81f"
server_key_version: "encryptionkey@0"
I20260923 15:02:10.957949 21768 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175691491353-13614-0/minicluster-data/master-0/wal/instance:
uuid: "07e5a7e3b63e4a1fb0c992a164407352"
format_stamp: "Formatted at 2026-09-23 15:02:10 on dist-test-slave-cnb0"
server_key: "9c89b5e11cabfad988408c40a99426e6"
server_key_iv: "bb2440d565f72ce876e51b5a26c3e81f"
server_key_version: "encryptionkey@0"
I20260923 15:02:10.959738 21768 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20260923 15:02:10.960745 21783 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:02:10.961107 21768 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:02:10.961246 21768 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175691491353-13614-0/minicluster-data/master-0/data,/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175691491353-13614-0/minicluster-data/master-0/wal
uuid: "07e5a7e3b63e4a1fb0c992a164407352"
format_stamp: "Formatted at 2026-09-23 15:02:10 on dist-test-slave-cnb0"
server_key: "9c89b5e11cabfad988408c40a99426e6"
server_key_iv: "bb2440d565f72ce876e51b5a26c3e81f"
server_key_version: "encryptionkey@0"
I20260923 15:02:10.961424 21768 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175691491353-13614-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175691491353-13614-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175691491353-13614-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:02:10.978680 21768 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:02:10.981256 21768 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:02:10.981436 21768 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:02:10.986159 21768 rpc_server.cc:307] RPC server started. Bound to: 127.13.75.190:40153
I20260923 15:02:10.986210 21835 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.75.190:40153 every 8 connection(s)
I20260923 15:02:10.986601 21768 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175691491353-13614-0/minicluster-data/master-0/data/info.pb
I20260923 15:02:10.987628 21836 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:02:10.990079 21836 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 07e5a7e3b63e4a1fb0c992a164407352: Bootstrap starting.
I20260923 15:02:10.990764 21836 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 07e5a7e3b63e4a1fb0c992a164407352: Neither blocks nor log segments found. Creating new log.
I20260923 15:02:10.991117 21836 log.cc:824] T 00000000000000000000000000000000 P 07e5a7e3b63e4a1fb0c992a164407352: Log is configured to *not* fsync() on all Append() calls
I20260923 15:02:10.992027 21836 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 07e5a7e3b63e4a1fb0c992a164407352: No bootstrap required, opened a new log
I20260923 15:02:10.993791 13614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKC4soy/build/release/bin/kudu as pid 21768
I20260923 15:02:10.993913 13614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175691491353-13614-0/minicluster-data/master-0/wal/instance
I20260923 15:02:10.993911 21836 raft_consensus.cc:348] T 00000000000000000000000000000000 P 07e5a7e3b63e4a1fb0c992a164407352 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "07e5a7e3b63e4a1fb0c992a164407352" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 40153 } }
I20260923 15:02:10.994095 13614 external_mini_cluster.cc:1428] Setting key b6a39fcb3681d0f3a26aa66a83be0ccc
I20260923 15:02:10.994133 21836 raft_consensus.cc:374] T 00000000000000000000000000000000 P 07e5a7e3b63e4a1fb0c992a164407352 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:02:10.994175 21836 raft_consensus.cc:729] T 00000000000000000000000000000000 P 07e5a7e3b63e4a1fb0c992a164407352 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07e5a7e3b63e4a1fb0c992a164407352, State: Initialized, Role: FOLLOWER
I20260923 15:02:10.994340 21836 consensus_queue.cc:260] T 00000000000000000000000000000000 P 07e5a7e3b63e4a1fb0c992a164407352 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "07e5a7e3b63e4a1fb0c992a164407352" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 40153 } }
I20260923 15:02:10.994438 21836 raft_consensus.cc:388] T 00000000000000000000000000000000 P 07e5a7e3b63e4a1fb0c992a164407352 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 15:02:10.994485 21836 raft_consensus.cc:482] T 00000000000000000000000000000000 P 07e5a7e3b63e4a1fb0c992a164407352 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 15:02:10.994572 21836 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 07e5a7e3b63e4a1fb0c992a164407352 [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:02:10.995931 21836 raft_consensus.cc:504] T 00000000000000000000000000000000 P 07e5a7e3b63e4a1fb0c992a164407352 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "07e5a7e3b63e4a1fb0c992a164407352" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 40153 } }
I20260923 15:02:10.996097 21836 leader_election.cc:302] T 00000000000000000000000000000000 P 07e5a7e3b63e4a1fb0c992a164407352 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 07e5a7e3b63e4a1fb0c992a164407352; no voters:
I20260923 15:02:10.996326 21836 leader_election.cc:288] T 00000000000000000000000000000000 P 07e5a7e3b63e4a1fb0c992a164407352 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 15:02:10.996407 21841 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 07e5a7e3b63e4a1fb0c992a164407352 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 15:02:10.996637 21836 sys_catalog.cc:564] T 00000000000000000000000000000000 P 07e5a7e3b63e4a1fb0c992a164407352 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 15:02:10.996796 21841 raft_consensus.cc:686] T 00000000000000000000000000000000 P 07e5a7e3b63e4a1fb0c992a164407352 [term 1 LEADER]: Becoming Leader. State: Replica: 07e5a7e3b63e4a1fb0c992a164407352, State: Running, Role: LEADER
I20260923 15:02:10.996999 21841 consensus_queue.cc:237] T 00000000000000000000000000000000 P 07e5a7e3b63e4a1fb0c992a164407352 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "07e5a7e3b63e4a1fb0c992a164407352" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 40153 } }
I20260923 15:02:10.997753 21843 sys_catalog.cc:455] T 00000000000000000000000000000000 P 07e5a7e3b63e4a1fb0c992a164407352 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 07e5a7e3b63e4a1fb0c992a164407352. Latest consensus state: current_term: 1 leader_uuid: "07e5a7e3b63e4a1fb0c992a164407352" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "07e5a7e3b63e4a1fb0c992a164407352" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 40153 } } }
I20260923 15:02:10.997864 21843 sys_catalog.cc:458] T 00000000000000000000000000000000 P 07e5a7e3b63e4a1fb0c992a164407352 [sys.catalog]: This master's current role is: LEADER
I20260923 15:02:10.997922 21842 sys_catalog.cc:455] T 00000000000000000000000000000000 P 07e5a7e3b63e4a1fb0c992a164407352 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "07e5a7e3b63e4a1fb0c992a164407352" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "07e5a7e3b63e4a1fb0c992a164407352" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 40153 } } }
I20260923 15:02:10.998102 21842 sys_catalog.cc:458] T 00000000000000000000000000000000 P 07e5a7e3b63e4a1fb0c992a164407352 [sys.catalog]: This master's current role is: LEADER
I20260923 15:02:10.998231 21850 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 15:02:10.998821 21850 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 15:02:11.001474 21850 catalog_manager.cc:1347] Generated new cluster ID: a771bcfb6fcb4bdaab18d81c5da4063e
I20260923 15:02:11.001556 21850 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 23 15:02:11 dist-test-slave-cnb0 krb5kdc[21755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175730, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.75.190@KRBTEST.COM
I20260923 15:02:11.008177 21840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:10.994643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48098 (local address 127.13.75.190:40153)
0923 15:02:10.995130 (+ 487us) server_negotiation.cc:206] Beginning negotiation
0923 15:02:10.995139 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0923 15:02:10.995169 (+ 30us) server_negotiation.cc:407] Connection header received
0923 15:02:10.995397 (+ 228us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:02:10.995406 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:02:10.995540 (+ 134us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:02:10.995732 (+ 192us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:02:10.996997 (+ 1265us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:10.998658 (+ 1661us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:11.000030 (+ 1372us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:11.000892 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:11.001090 (+ 198us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:11.004218 (+ 3128us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:02:11.004223 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:02:11.004223 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:02:11.004235 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:02:11.006294 (+ 2059us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:02:11.006778 (+ 484us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:02:11.006782 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:02:11.006783 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:02:11.006973 (+ 190us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:02:11.007209 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:02:11.007209 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:02:11.007210 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:02:11.007499 (+ 289us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:02:11.007677 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0923 15:02:11.007793 (+ 116us) server_negotiation.cc:299] Negotiation successful
0923 15:02:11.007952 (+ 159us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":431,"thread_start_us":105,"threads_started":1}
I20260923 15:02:11.023293 21850 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 15:02:11.023921 21850 catalog_manager.cc:1504] Loading token signing keys...
I20260923 15:02:11.034627 21850 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 07e5a7e3b63e4a1fb0c992a164407352: Generated new TSK 0
I20260923 15:02:11.034919 21850 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.75.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.75.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.75.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.129.keytab.
Entry for principal kudu/127.13.75.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.129.keytab.
I20260923 15:02:11.067853 13614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKC4soy/build/release/bin/kudu
/tmp/dist-test-taskKC4soy/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175691491353-13614-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175691491353-13614-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175691491353-13614-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175691491353-13614-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.75.129:0
--local_ip_for_outbound_sockets=127.13.75.129
--webserver_interface=127.13.75.129
--webserver_port=0
--tserver_master_addrs=127.13.75.190:40153
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.129.keytab
--principal=kudu/127.13.75.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.75.148:35183
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175691491353-13614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175691491353-13614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175691491353-13614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:02:11.155934 21862 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:02:11.156239 21862 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:02:11.156286 21862 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:02:11.157660 21862 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:02:11.157740 21862 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:02:11.157794 21862 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.75.129
I20260923 15:02:11.159567 21862 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.75.148:35183
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175691491353-13614-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175691491353-13614-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.129.keytab
--principal=kudu/127.13.75.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.75.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175691491353-13614-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.75.129
--webserver_port=0
--tserver_master_addrs=127.13.75.190:40153
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21862
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.75.129
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175691491353-13614-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:02:11.159899 21862 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:02:11.160154 21862 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:02:11.165971 21868 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:02:11.165999 21867 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:02:11.165971 21870 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:02:11.166225 21862 server_base.cc:1034] running on GCE node
I20260923 15:02:11.166530 21862 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:02:11.167001 21862 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:02:11.168262 21862 hybrid_clock.cc:648] HybridClock initialized: now 1790175731168249 us; error 40 us; skew 500 ppm
Sep 23 15:02:11 dist-test-slave-cnb0 krb5kdc[21755](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175731, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:02:11.171334 21862 init.cc:377] Logged in from keytab as kudu/127.13.75.129@KRBTEST.COM (short username kudu)
I20260923 15:02:11.171926 21862 webserver.cc:460] Webserver started at http://127.13.75.129:44351/ using document root <none> and password file <none>
I20260923 15:02:11.172188 21862 fs_manager.cc:359] Metadata directory not provided
I20260923 15:02:11.172259 21862 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:02:11.172415 21862 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:02:11.173520 21862 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175691491353-13614-0/minicluster-data/ts-0/data/instance:
uuid: "88544b8d6eb5433dbaa9980bca0fbda3"
format_stamp: "Formatted at 2026-09-23 15:02:11 on dist-test-slave-cnb0"
server_key: "0138b7cd0f76181a32320b4ce763d7d2"
server_key_iv: "97ccdc288f4c33b522f4a96c9d4c877c"
server_key_version: "encryptionkey@0"
I20260923 15:02:11.173890 21862 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175691491353-13614-0/minicluster-data/ts-0/wal/instance:
uuid: "88544b8d6eb5433dbaa9980bca0fbda3"
format_stamp: "Formatted at 2026-09-23 15:02:11 on dist-test-slave-cnb0"
server_key: "0138b7cd0f76181a32320b4ce763d7d2"
server_key_iv: "97ccdc288f4c33b522f4a96c9d4c877c"
server_key_version: "encryptionkey@0"
I20260923 15:02:11.175427 21862 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 15:02:11.176563 21877 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:02:11.176800 21862 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:02:11.176927 21862 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175691491353-13614-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175691491353-13614-0/minicluster-data/ts-0/wal
uuid: "88544b8d6eb5433dbaa9980bca0fbda3"
format_stamp: "Formatted at 2026-09-23 15:02:11 on dist-test-slave-cnb0"
server_key: "0138b7cd0f76181a32320b4ce763d7d2"
server_key_iv: "97ccdc288f4c33b522f4a96c9d4c877c"
server_key_version: "encryptionkey@0"
I20260923 15:02:11.177014 21862 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175691491353-13614-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175691491353-13614-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175691491353-13614-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:02:11.197535 21862 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:02:11.200351 21862 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:02:11.200621 21862 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:02:11.200888 21862 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 15:02:11.201246 21862 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 15:02:11.201306 21862 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 15:02:11.201349 21862 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 15:02:11.201378 21862 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 15:02:11.207759 21862 rpc_server.cc:307] RPC server started. Bound to: 127.13.75.129:32801
I20260923 15:02:11.207840 21991 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.75.129:32801 every 8 connection(s)
I20260923 15:02:11.208235 21862 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175691491353-13614-0/minicluster-data/ts-0/data/info.pb
I20260923 15:02:11.214329 13614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKC4soy/build/release/bin/kudu as pid 21862
I20260923 15:02:11.214421 13614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175691491353-13614-0/minicluster-data/ts-0/wal/instance
I20260923 15:02:11.214645 13614 external_mini_cluster.cc:1428] Setting key 2b129de7255c323018182166cd49fdf8
Sep 23 15:02:11 dist-test-slave-cnb0 krb5kdc[21755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175731, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.129@KRBTEST.COM for kudu/127.13.75.190@KRBTEST.COM
I20260923 15:02:11.219293 21840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:11.209177 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.129:47405 (local address 127.13.75.190:40153)
0923 15:02:11.209258 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0923 15:02:11.209260 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:02:11.209484 (+ 224us) server_negotiation.cc:407] Connection header received
0923 15:02:11.209984 (+ 500us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:02:11.209985 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:02:11.210000 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:02:11.210066 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:02:11.211221 (+ 1155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:11.211788 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:11.212897 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:11.213110 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:11.213180 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:11.215799 (+ 2619us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:02:11.215801 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:02:11.215802 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:02:11.215810 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:02:11.217468 (+ 1658us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:02:11.218126 (+ 658us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:02:11.218127 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:02:11.218127 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:02:11.218203 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:02:11.218524 (+ 321us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:02:11.218525 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:02:11.218525 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:02:11.218667 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:02:11.218734 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0923 15:02:11.219133 (+ 399us) server_negotiation.cc:299] Negotiation successful
0923 15:02:11.219222 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260923 15:02:11.219619 21994 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:11.209088 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:40153 (local address 127.13.75.129:47405)
0923 15:02:11.209355 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0923 15:02:11.209396 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0923 15:02:11.209862 (+ 466us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:02:11.210123 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:02:11.210127 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:02:11.210729 (+ 602us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:02:11.211029 (+ 300us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:11.211035 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:11.211902 (+ 867us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:11.211903 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:11.212787 (+ 884us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:11.212790 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:11.213219 (+ 429us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:11.213220 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:11.213337 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:11.213893 (+ 556us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:02:11.213902 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:02:11.215696 (+ 1794us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:02:11.217565 (+ 1869us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:02:11.217575 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:02:11.217578 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:02:11.217976 (+ 398us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:02:11.218291 (+ 315us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:02:11.218292 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:02:11.218293 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:02:11.218452 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:02:11.218740 (+ 288us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:02:11.218748 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:02:11.218970 (+ 222us) client_negotiation.cc:769] Sending connection context
0923 15:02:11.219101 (+ 131us) client_negotiation.cc:240] Negotiation successful
0923 15:02:11.219302 (+ 201us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":223,"thread_start_us":121,"threads_started":1}
I20260923 15:02:11.220219 21992 heartbeater.cc:347] Connected to a master server at 127.13.75.190:40153
I20260923 15:02:11.220335 21992 heartbeater.cc:464] Registering TS with master...
I20260923 15:02:11.220561 21992 heartbeater.cc:511] Master 127.13.75.190:40153 requested a full tablet report, sending...
I20260923 15:02:11.221135 21800 ts_manager.cc:194] Registered new tserver with Master: 88544b8d6eb5433dbaa9980bca0fbda3 (127.13.75.129:32801)
I20260923 15:02:11.221766 21800 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.75.129@KRBTEST.COM'} at 127.13.75.129:47405
WARNING: no policy specified for kudu/127.13.75.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.75.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.75.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.130.keytab.
Entry for principal kudu/127.13.75.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.130.keytab.
I20260923 15:02:11.246580 13614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKC4soy/build/release/bin/kudu
/tmp/dist-test-taskKC4soy/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175691491353-13614-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175691491353-13614-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175691491353-13614-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175691491353-13614-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.75.130:0
--local_ip_for_outbound_sockets=127.13.75.130
--webserver_interface=127.13.75.130
--webserver_port=0
--tserver_master_addrs=127.13.75.190:40153
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.130.keytab
--principal=kudu/127.13.75.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.75.148:35183
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175691491353-13614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175691491353-13614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175691491353-13614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:02:11.335465 21997 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:02:11.335736 21997 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:02:11.335765 21997 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:02:11.337246 21997 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:02:11.337319 21997 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:02:11.337352 21997 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.75.130
I20260923 15:02:11.339373 21997 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.75.148:35183
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175691491353-13614-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175691491353-13614-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.130.keytab
--principal=kudu/127.13.75.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.75.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175691491353-13614-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.75.130
--webserver_port=0
--tserver_master_addrs=127.13.75.190:40153
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.21997
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.75.130
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175691491353-13614-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:02:11.339697 21997 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:02:11.339962 21997 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:02:11.345741 22003 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:02:11.345750 22002 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:02:11.345736 22005 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:02:11.345899 21997 server_base.cc:1034] running on GCE node
I20260923 15:02:11.346210 21997 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:02:11.346480 21997 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:02:11.347642 21997 hybrid_clock.cc:648] HybridClock initialized: now 1790175731347600 us; error 52 us; skew 500 ppm
Sep 23 15:02:11 dist-test-slave-cnb0 krb5kdc[21755](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175731, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:02:11.350454 21997 init.cc:377] Logged in from keytab as kudu/127.13.75.130@KRBTEST.COM (short username kudu)
I20260923 15:02:11.350975 21997 webserver.cc:460] Webserver started at http://127.13.75.130:38093/ using document root <none> and password file <none>
I20260923 15:02:11.351187 21997 fs_manager.cc:359] Metadata directory not provided
I20260923 15:02:11.351231 21997 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:02:11.351352 21997 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:02:11.352394 21997 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175691491353-13614-0/minicluster-data/ts-1/data/instance:
uuid: "13e655804acf4f37a894d688026ed923"
format_stamp: "Formatted at 2026-09-23 15:02:11 on dist-test-slave-cnb0"
server_key: "23233fa036f2316b56ee108d45140e29"
server_key_iv: "ac9d889d05de5db6fd6b56af408628e0"
server_key_version: "encryptionkey@0"
I20260923 15:02:11.352717 21997 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175691491353-13614-0/minicluster-data/ts-1/wal/instance:
uuid: "13e655804acf4f37a894d688026ed923"
format_stamp: "Formatted at 2026-09-23 15:02:11 on dist-test-slave-cnb0"
server_key: "23233fa036f2316b56ee108d45140e29"
server_key_iv: "ac9d889d05de5db6fd6b56af408628e0"
server_key_version: "encryptionkey@0"
I20260923 15:02:11.354113 21997 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260923 15:02:11.355026 22012 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:02:11.355243 21997 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260923 15:02:11.355357 21997 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175691491353-13614-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175691491353-13614-0/minicluster-data/ts-1/wal
uuid: "13e655804acf4f37a894d688026ed923"
format_stamp: "Formatted at 2026-09-23 15:02:11 on dist-test-slave-cnb0"
server_key: "23233fa036f2316b56ee108d45140e29"
server_key_iv: "ac9d889d05de5db6fd6b56af408628e0"
server_key_version: "encryptionkey@0"
I20260923 15:02:11.355419 21997 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175691491353-13614-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175691491353-13614-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175691491353-13614-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:02:11.391475 21997 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:02:11.393832 21997 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:02:11.393958 21997 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:02:11.394161 21997 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 15:02:11.394531 21997 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 15:02:11.394569 21997 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 15:02:11.394593 21997 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 15:02:11.394605 21997 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 15:02:11.400816 21997 rpc_server.cc:307] RPC server started. Bound to: 127.13.75.130:33961
I20260923 15:02:11.400876 22125 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.75.130:33961 every 8 connection(s)
I20260923 15:02:11.401208 21997 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175691491353-13614-0/minicluster-data/ts-1/data/info.pb
I20260923 15:02:11.402922 13614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKC4soy/build/release/bin/kudu as pid 21997
I20260923 15:02:11.403051 13614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175691491353-13614-0/minicluster-data/ts-1/wal/instance
I20260923 15:02:11.403244 13614 external_mini_cluster.cc:1428] Setting key 0909158a1cd81b417cc43aa76f3e2403
Sep 23 15:02:11 dist-test-slave-cnb0 krb5kdc[21755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175731, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.130@KRBTEST.COM for kudu/127.13.75.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.13.75.131@KRBTEST.COM; defaulting to no policy
I20260923 15:02:11.412514 22128 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:11.401980 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:40153 (local address 127.13.75.130:37531)
0923 15:02:11.402175 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0923 15:02:11.402227 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0923 15:02:11.402796 (+ 569us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:02:11.403075 (+ 279us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:02:11.403079 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:02:11.403824 (+ 745us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:02:11.404117 (+ 293us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:11.404124 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:11.404822 (+ 698us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:11.404823 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:11.406112 (+ 1289us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:11.406116 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:11.406585 (+ 469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:11.406587 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:11.406750 (+ 163us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:11.407360 (+ 610us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:02:11.407372 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:02:11.409076 (+ 1704us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:02:11.410854 (+ 1778us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:02:11.410863 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:02:11.410866 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:02:11.411170 (+ 304us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:02:11.411395 (+ 225us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:02:11.411395 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:02:11.411396 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:02:11.411520 (+ 124us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:02:11.411828 (+ 308us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:02:11.411836 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:02:11.412039 (+ 203us) client_negotiation.cc:769] Sending connection context
0923 15:02:11.412160 (+ 121us) client_negotiation.cc:240] Negotiation successful
0923 15:02:11.412303 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":93,"threads_started":1}
I20260923 15:02:11.412520 21840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:11.402026 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.130:37531 (local address 127.13.75.190:40153)
0923 15:02:11.402125 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0923 15:02:11.402128 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:02:11.402324 (+ 196us) server_negotiation.cc:407] Connection header received
0923 15:02:11.402931 (+ 607us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:02:11.402932 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:02:11.402945 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:02:11.402986 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:02:11.404216 (+ 1230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:11.404711 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:11.406271 (+ 1560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:11.406484 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:11.406565 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:11.409195 (+ 2630us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:02:11.409198 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:02:11.409198 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:02:11.409207 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:02:11.410737 (+ 1530us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:02:11.411258 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:02:11.411260 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:02:11.411260 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:02:11.411324 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:02:11.411604 (+ 280us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:02:11.411605 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:02:11.411606 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:02:11.411746 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:02:11.411812 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0923 15:02:11.412319 (+ 507us) server_negotiation.cc:299] Negotiation successful
0923 15:02:11.412454 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":47}
I20260923 15:02:11.413314 22126 heartbeater.cc:347] Connected to a master server at 127.13.75.190:40153
I20260923 15:02:11.413409 22126 heartbeater.cc:464] Registering TS with master...
I20260923 15:02:11.413589 22126 heartbeater.cc:511] Master 127.13.75.190:40153 requested a full tablet report, sending...
I20260923 15:02:11.413988 21800 ts_manager.cc:194] Registered new tserver with Master: 13e655804acf4f37a894d688026ed923 (127.13.75.130:33961)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.75.131@KRBTEST.COM" created.
I20260923 15:02:11.414463 21800 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.75.130@KRBTEST.COM'} at 127.13.75.130:37531
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.75.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.131.keytab.
Entry for principal kudu/127.13.75.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.131.keytab.
I20260923 15:02:11.431627 13614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKC4soy/build/release/bin/kudu
/tmp/dist-test-taskKC4soy/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175691491353-13614-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175691491353-13614-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175691491353-13614-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175691491353-13614-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.75.131:0
--local_ip_for_outbound_sockets=127.13.75.131
--webserver_interface=127.13.75.131
--webserver_port=0
--tserver_master_addrs=127.13.75.190:40153
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.131.keytab
--principal=kudu/127.13.75.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.75.148:35183
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175691491353-13614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175691491353-13614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175691491353-13614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:02:11.517609 22131 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:02:11.517838 22131 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:02:11.517864 22131 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:02:11.519170 22131 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:02:11.519222 22131 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:02:11.519251 22131 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.75.131
I20260923 15:02:11.520757 22131 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.75.148:35183
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175691491353-13614-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175691491353-13614-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.131.keytab
--principal=kudu/127.13.75.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.75.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175691491353-13614-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.75.131
--webserver_port=0
--tserver_master_addrs=127.13.75.190:40153
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22131
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.75.131
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175691491353-13614-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:02:11.521041 22131 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:02:11.521289 22131 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:02:11.526716 22139 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:02:11.526731 22137 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:02:11.526800 22131 server_base.cc:1034] running on GCE node
W20260923 15:02:11.526731 22136 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:02:11.527218 22131 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:02:11.527551 22131 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:02:11.528748 22131 hybrid_clock.cc:648] HybridClock initialized: now 1790175731528720 us; error 39 us; skew 500 ppm
Sep 23 15:02:11 dist-test-slave-cnb0 krb5kdc[21755](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175731, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:02:11.531765 22131 init.cc:377] Logged in from keytab as kudu/127.13.75.131@KRBTEST.COM (short username kudu)
I20260923 15:02:11.532282 22131 webserver.cc:460] Webserver started at http://127.13.75.131:45091/ using document root <none> and password file <none>
I20260923 15:02:11.532548 22131 fs_manager.cc:359] Metadata directory not provided
I20260923 15:02:11.532599 22131 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:02:11.532750 22131 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:02:11.534001 22131 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175691491353-13614-0/minicluster-data/ts-2/data/instance:
uuid: "4d63b5f9ed9d477bb24b09f7498aebaa"
format_stamp: "Formatted at 2026-09-23 15:02:11 on dist-test-slave-cnb0"
server_key: "a954a76a88a427cf434eef5ef8704e5c"
server_key_iv: "58e9ab407c098fbb240a5ad4e380c3a9"
server_key_version: "encryptionkey@0"
I20260923 15:02:11.534390 22131 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175691491353-13614-0/minicluster-data/ts-2/wal/instance:
uuid: "4d63b5f9ed9d477bb24b09f7498aebaa"
format_stamp: "Formatted at 2026-09-23 15:02:11 on dist-test-slave-cnb0"
server_key: "a954a76a88a427cf434eef5ef8704e5c"
server_key_iv: "58e9ab407c098fbb240a5ad4e380c3a9"
server_key_version: "encryptionkey@0"
I20260923 15:02:11.535997 22131 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 15:02:11.536988 22146 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:02:11.537253 22131 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:02:11.537433 22131 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175691491353-13614-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175691491353-13614-0/minicluster-data/ts-2/wal
uuid: "4d63b5f9ed9d477bb24b09f7498aebaa"
format_stamp: "Formatted at 2026-09-23 15:02:11 on dist-test-slave-cnb0"
server_key: "a954a76a88a427cf434eef5ef8704e5c"
server_key_iv: "58e9ab407c098fbb240a5ad4e380c3a9"
server_key_version: "encryptionkey@0"
I20260923 15:02:11.537524 22131 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175691491353-13614-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175691491353-13614-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175691491353-13614-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:02:11.552655 22131 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:02:11.555207 22131 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:02:11.555402 22131 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:02:11.555681 22131 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 15:02:11.556082 22131 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 15:02:11.556149 22131 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 15:02:11.556196 22131 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 15:02:11.556248 22131 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 15:02:11.563063 22131 rpc_server.cc:307] RPC server started. Bound to: 127.13.75.131:35399
I20260923 15:02:11.563108 22259 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.75.131:35399 every 8 connection(s)
I20260923 15:02:11.563498 22131 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175691491353-13614-0/minicluster-data/ts-2/data/info.pb
I20260923 15:02:11.567255 13614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKC4soy/build/release/bin/kudu as pid 22131
I20260923 15:02:11.567471 13614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175691491353-13614-0/minicluster-data/ts-2/wal/instance
I20260923 15:02:11.567646 13614 external_mini_cluster.cc:1428] Setting key 837e8d40a28e0de56964c574d25a6476
Sep 23 15:02:11 dist-test-slave-cnb0 krb5kdc[21755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175731, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.131@KRBTEST.COM for kudu/127.13.75.190@KRBTEST.COM
I20260923 15:02:11.573798 21840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:11.564335 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.131:49783 (local address 127.13.75.190:40153)
0923 15:02:11.564468 (+ 133us) server_negotiation.cc:206] Beginning negotiation
0923 15:02:11.564471 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:02:11.564629 (+ 158us) server_negotiation.cc:407] Connection header received
0923 15:02:11.565187 (+ 558us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:02:11.565188 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:02:11.565199 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:02:11.565263 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:02:11.566288 (+ 1025us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:11.566830 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:11.568118 (+ 1288us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:11.568433 (+ 315us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:11.568502 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:11.570760 (+ 2258us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:02:11.570761 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:02:11.570762 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:02:11.570769 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:02:11.572308 (+ 1539us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:02:11.572772 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:02:11.572773 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:02:11.572773 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:02:11.572827 (+ 54us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:02:11.573074 (+ 247us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:02:11.573075 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:02:11.573075 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:02:11.573185 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:02:11.573238 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0923 15:02:11.573625 (+ 387us) server_negotiation.cc:299] Negotiation successful
0923 15:02:11.573708 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260923 15:02:11.573966 22262 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:11.564300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:40153 (local address 127.13.75.131:49783)
0923 15:02:11.564484 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0923 15:02:11.564536 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0923 15:02:11.565103 (+ 567us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:02:11.565328 (+ 225us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:02:11.565332 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:02:11.565965 (+ 633us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:02:11.566166 (+ 201us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:11.566171 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:11.567000 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:11.567001 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:11.567999 (+ 998us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:11.568002 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:11.568515 (+ 513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:11.568516 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:11.568708 (+ 192us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:11.569207 (+ 499us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:02:11.569216 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:02:11.570671 (+ 1455us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:02:11.572412 (+ 1741us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:02:11.572419 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:02:11.572421 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:02:11.572699 (+ 278us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:02:11.572908 (+ 209us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:02:11.572908 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:02:11.572909 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:02:11.573016 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:02:11.573268 (+ 252us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:02:11.573274 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:02:11.573456 (+ 182us) client_negotiation.cc:769] Sending connection context
0923 15:02:11.573563 (+ 107us) client_negotiation.cc:240] Negotiation successful
0923 15:02:11.573715 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":109,"threads_started":1}
I20260923 15:02:11.574616 22260 heartbeater.cc:347] Connected to a master server at 127.13.75.190:40153
I20260923 15:02:11.574753 22260 heartbeater.cc:464] Registering TS with master...
I20260923 15:02:11.575040 22260 heartbeater.cc:511] Master 127.13.75.190:40153 requested a full tablet report, sending...
I20260923 15:02:11.575570 21800 ts_manager.cc:194] Registered new tserver with Master: 4d63b5f9ed9d477bb24b09f7498aebaa (127.13.75.131:35399)
I20260923 15:02:11.575929 21800 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.75.131@KRBTEST.COM'} at 127.13.75.131:49783
I20260923 15:02:11.579808 13614 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
WARNING: no policy specified for oryx/127.13.75.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.13.75.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.13.75.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175691491353-13614-0/krb5kdc/oryx_127.13.75.129.keytab.
Entry for principal oryx/127.13.75.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175691491353-13614-0/krb5kdc/oryx_127.13.75.129.keytab.
WARNING: no policy specified for oryx/127.13.75.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.13.75.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.13.75.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175691491353-13614-0/krb5kdc/oryx_127.13.75.130.keytab.
Entry for principal oryx/127.13.75.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175691491353-13614-0/krb5kdc/oryx_127.13.75.130.keytab.
WARNING: no policy specified for oryx/127.13.75.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "oryx/127.13.75.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal oryx/127.13.75.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175691491353-13614-0/krb5kdc/oryx_127.13.75.131.keytab.
Entry for principal oryx/127.13.75.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175691491353-13614-0/krb5kdc/oryx_127.13.75.131.keytab.
I20260923 15:02:11.665802 13614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKC4soy/build/release/bin/kudu with pid 21862
I20260923 15:02:11.672151 13614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKC4soy/build/release/bin/kudu with pid 21997
I20260923 15:02:11.679548 13614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKC4soy/build/release/bin/kudu with pid 22131
I20260923 15:02:11.684517 13614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKC4soy/build/release/bin/kudu with pid 21768
I20260923 15:02:11.690026 13614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKC4soy/build/release/bin/kudu
/tmp/dist-test-taskKC4soy/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175691491353-13614-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175691491353-13614-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175691491353-13614-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175691491353-13614-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.75.190:40153
--webserver_interface=127.13.75.190
--webserver_port=34373
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.190.keytab
--principal=kudu/127.13.75.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.75.148:35183
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.75.190:40153
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175691491353-13614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175691491353-13614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175691491353-13614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:02:11.781034 22269 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:02:11.781280 22269 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:02:11.781306 22269 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:02:11.783061 22269 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 15:02:11.783141 22269 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:02:11.783159 22269 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:02:11.783174 22269 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 15:02:11.783187 22269 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 15:02:11.785179 22269 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.75.148:35183
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175691491353-13614-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175691491353-13614-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.75.190:40153
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.190.keytab
--principal=kudu/127.13.75.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.75.190:40153
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175691491353-13614-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.75.190
--webserver_port=34373
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22269
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175691491353-13614-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:02:11.785609 22269 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:02:11.785952 22269 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:02:11.791440 22277 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:02:11.791520 22275 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:02:11.791440 22274 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:02:11.791916 22269 server_base.cc:1034] running on GCE node
I20260923 15:02:11.792289 22269 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:02:11.792594 22269 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:02:11.793738 22269 hybrid_clock.cc:648] HybridClock initialized: now 1790175731793721 us; error 39 us; skew 500 ppm
Sep 23 15:02:11 dist-test-slave-cnb0 krb5kdc[21755](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175731, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:02:11.796559 22269 init.cc:377] Logged in from keytab as kudu/127.13.75.190@KRBTEST.COM (short username kudu)
I20260923 15:02:11.797063 22269 webserver.cc:460] Webserver started at http://127.13.75.190:34373/ using document root <none> and password file <none>
I20260923 15:02:11.797312 22269 fs_manager.cc:359] Metadata directory not provided
I20260923 15:02:11.797407 22269 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:02:11.798998 22269 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 15:02:11.799834 22284 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:02:11.800187 22269 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:02:11.800307 22269 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175691491353-13614-0/minicluster-data/master-0/data,/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175691491353-13614-0/minicluster-data/master-0/wal
uuid: "07e5a7e3b63e4a1fb0c992a164407352"
format_stamp: "Formatted at 2026-09-23 15:02:10 on dist-test-slave-cnb0"
server_key: "9c89b5e11cabfad988408c40a99426e6"
server_key_iv: "bb2440d565f72ce876e51b5a26c3e81f"
server_key_version: "encryptionkey@0"
I20260923 15:02:11.800715 22269 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175691491353-13614-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175691491353-13614-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175691491353-13614-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:02:11.809675 22269 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:02:11.811988 22269 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:02:11.812301 22269 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:02:11.816910 22269 rpc_server.cc:307] RPC server started. Bound to: 127.13.75.190:40153
I20260923 15:02:11.816962 22336 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.75.190:40153 every 8 connection(s)
I20260923 15:02:11.817364 22269 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175691491353-13614-0/minicluster-data/master-0/data/info.pb
I20260923 15:02:11.819144 22337 sys_catalog.cc:263] Verifying existing consensus state
I20260923 15:02:11.819957 22337 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 07e5a7e3b63e4a1fb0c992a164407352: Bootstrap starting.
I20260923 15:02:11.826082 13614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKC4soy/build/release/bin/kudu as pid 22269
I20260923 15:02:11.826439 13614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKC4soy/build/release/bin/kudu
/tmp/dist-test-taskKC4soy/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175691491353-13614-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175691491353-13614-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175691491353-13614-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175691491353-13614-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.75.129:32801
--local_ip_for_outbound_sockets=127.13.75.129
--tserver_master_addrs=127.13.75.190:40153
--webserver_port=44351
--webserver_interface=127.13.75.129
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.129.keytab
--principal=kudu/127.13.75.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.75.148:35183
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.13.75.129
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175691491353-13614-0/krb5kdc/oryx_127.13.75.129.keytab with env {KRB5CCNAME=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175691491353-13614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175691491353-13614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175691491353-13614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260923 15:02:11.829466 22337 log.cc:824] T 00000000000000000000000000000000 P 07e5a7e3b63e4a1fb0c992a164407352: Log is configured to *not* fsync() on all Append() calls
I20260923 15:02:11.831460 22337 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 07e5a7e3b63e4a1fb0c992a164407352: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260923 15:02:11.831772 22337 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 07e5a7e3b63e4a1fb0c992a164407352: Bootstrap complete.
I20260923 15:02:11.834157 22337 raft_consensus.cc:348] T 00000000000000000000000000000000 P 07e5a7e3b63e4a1fb0c992a164407352 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "07e5a7e3b63e4a1fb0c992a164407352" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 40153 } }
I20260923 15:02:11.834429 22337 raft_consensus.cc:729] T 00000000000000000000000000000000 P 07e5a7e3b63e4a1fb0c992a164407352 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 07e5a7e3b63e4a1fb0c992a164407352, State: Initialized, Role: FOLLOWER
I20260923 15:02:11.834643 22337 consensus_queue.cc:260] T 00000000000000000000000000000000 P 07e5a7e3b63e4a1fb0c992a164407352 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "07e5a7e3b63e4a1fb0c992a164407352" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 40153 } }
I20260923 15:02:11.834743 22337 raft_consensus.cc:388] T 00000000000000000000000000000000 P 07e5a7e3b63e4a1fb0c992a164407352 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 15:02:11.834797 22337 raft_consensus.cc:482] T 00000000000000000000000000000000 P 07e5a7e3b63e4a1fb0c992a164407352 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 15:02:11.834846 22337 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 07e5a7e3b63e4a1fb0c992a164407352 [term 1 FOLLOWER]: Advancing to term 2
I20260923 15:02:11.835945 22337 raft_consensus.cc:504] T 00000000000000000000000000000000 P 07e5a7e3b63e4a1fb0c992a164407352 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "07e5a7e3b63e4a1fb0c992a164407352" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 40153 } }
I20260923 15:02:11.836081 22337 leader_election.cc:302] T 00000000000000000000000000000000 P 07e5a7e3b63e4a1fb0c992a164407352 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 07e5a7e3b63e4a1fb0c992a164407352; no voters:
I20260923 15:02:11.836292 22337 leader_election.cc:288] T 00000000000000000000000000000000 P 07e5a7e3b63e4a1fb0c992a164407352 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260923 15:02:11.836337 22341 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 07e5a7e3b63e4a1fb0c992a164407352 [term 2 FOLLOWER]: Leader election won for term 2
I20260923 15:02:11.836459 22341 raft_consensus.cc:686] T 00000000000000000000000000000000 P 07e5a7e3b63e4a1fb0c992a164407352 [term 2 LEADER]: Becoming Leader. State: Replica: 07e5a7e3b63e4a1fb0c992a164407352, State: Running, Role: LEADER
I20260923 15:02:11.836585 22337 sys_catalog.cc:564] T 00000000000000000000000000000000 P 07e5a7e3b63e4a1fb0c992a164407352 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 15:02:11.836560 22341 consensus_queue.cc:237] T 00000000000000000000000000000000 P 07e5a7e3b63e4a1fb0c992a164407352 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "07e5a7e3b63e4a1fb0c992a164407352" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 40153 } }
I20260923 15:02:11.836860 22341 sys_catalog.cc:455] T 00000000000000000000000000000000 P 07e5a7e3b63e4a1fb0c992a164407352 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 07e5a7e3b63e4a1fb0c992a164407352. Latest consensus state: current_term: 2 leader_uuid: "07e5a7e3b63e4a1fb0c992a164407352" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "07e5a7e3b63e4a1fb0c992a164407352" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 40153 } } }
I20260923 15:02:11.836932 22341 sys_catalog.cc:458] T 00000000000000000000000000000000 P 07e5a7e3b63e4a1fb0c992a164407352 [sys.catalog]: This master's current role is: LEADER
I20260923 15:02:11.837085 22342 sys_catalog.cc:455] T 00000000000000000000000000000000 P 07e5a7e3b63e4a1fb0c992a164407352 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "07e5a7e3b63e4a1fb0c992a164407352" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "07e5a7e3b63e4a1fb0c992a164407352" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 40153 } } }
I20260923 15:02:11.837231 22342 sys_catalog.cc:458] T 00000000000000000000000000000000 P 07e5a7e3b63e4a1fb0c992a164407352 [sys.catalog]: This master's current role is: LEADER
I20260923 15:02:11.837163 22345 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 15:02:11.837843 22345 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 15:02:11.838143 22345 catalog_manager.cc:1259] Loaded cluster ID: a771bcfb6fcb4bdaab18d81c5da4063e
I20260923 15:02:11.838223 22345 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 15:02:11.839711 22345 catalog_manager.cc:1504] Loading token signing keys...
I20260923 15:02:11.839900 22345 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 07e5a7e3b63e4a1fb0c992a164407352: Loaded TSK: 0
I20260923 15:02:11.840123 22345 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260923 15:02:11.918702 22339 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:02:11.919037 22339 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:02:11.919072 22339 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:02:11.920495 22339 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:02:11.920552 22339 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:02:11.920581 22339 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.75.129
I20260923 15:02:11.922170 22339 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.75.148:35183
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175691491353-13614-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175691491353-13614-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175691491353-13614-0/krb5kdc/oryx_127.13.75.129.keytab
--principal=oryx/127.13.75.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.75.129:32801
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175691491353-13614-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.75.129
--webserver_port=44351
--tserver_master_addrs=127.13.75.190:40153
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22339
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.75.129
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175691491353-13614-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:02:11.922448 22339 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:02:11.922765 22339 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:02:11.928197 22364 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:02:11.928208 22363 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:02:11.928339 22366 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:02:11.928882 22339 server_base.cc:1034] running on GCE node
I20260923 15:02:11.929200 22339 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:02:11.929538 22339 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:02:11.930742 22339 hybrid_clock.cc:648] HybridClock initialized: now 1790175731930725 us; error 33 us; skew 500 ppm
Sep 23 15:02:11 dist-test-slave-cnb0 krb5kdc[21755](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175731, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.75.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:02:11.933960 22339 init.cc:377] Logged in from keytab as oryx/127.13.75.129@KRBTEST.COM (short username oryx)
I20260923 15:02:11.934536 22339 webserver.cc:460] Webserver started at http://127.13.75.129:44351/ using document root <none> and password file <none>
I20260923 15:02:11.934788 22339 fs_manager.cc:359] Metadata directory not provided
I20260923 15:02:11.934876 22339 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:02:11.936568 22339 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:02:11.937412 22373 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:02:11.937630 22339 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:02:11.937728 22339 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175691491353-13614-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175691491353-13614-0/minicluster-data/ts-0/wal
uuid: "88544b8d6eb5433dbaa9980bca0fbda3"
format_stamp: "Formatted at 2026-09-23 15:02:11 on dist-test-slave-cnb0"
server_key: "0138b7cd0f76181a32320b4ce763d7d2"
server_key_iv: "97ccdc288f4c33b522f4a96c9d4c877c"
server_key_version: "encryptionkey@0"
I20260923 15:02:11.938148 22339 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175691491353-13614-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175691491353-13614-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175691491353-13614-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:02:11.963968 22339 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:02:11.966416 22339 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:02:11.966611 22339 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:02:11.966871 22339 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 15:02:11.967267 22339 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 15:02:11.967324 22339 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 15:02:11.967366 22339 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 15:02:11.967401 22339 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 15:02:11.973392 22339 rpc_server.cc:307] RPC server started. Bound to: 127.13.75.129:32801
I20260923 15:02:11.973434 22486 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.75.129:32801 every 8 connection(s)
I20260923 15:02:11.973848 22339 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175691491353-13614-0/minicluster-data/ts-0/data/info.pb
Sep 23 15:02:11 dist-test-slave-cnb0 krb5kdc[21755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.75.129@KRBTEST.COM for oryx/127.13.75.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 15:02:11 dist-test-slave-cnb0 krb5kdc[21755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.75.129@KRBTEST.COM for oryx/127.13.75.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 15:02:11.983125 13614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKC4soy/build/release/bin/kudu as pid 22339
I20260923 15:02:11.983476 13614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKC4soy/build/release/bin/kudu
/tmp/dist-test-taskKC4soy/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175691491353-13614-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175691491353-13614-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175691491353-13614-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175691491353-13614-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.75.130:33961
--local_ip_for_outbound_sockets=127.13.75.130
--tserver_master_addrs=127.13.75.190:40153
--webserver_port=38093
--webserver_interface=127.13.75.130
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.130.keytab
--principal=kudu/127.13.75.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.75.148:35183
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.13.75.130
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175691491353-13614-0/krb5kdc/oryx_127.13.75.130.keytab with env {KRB5CCNAME=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175691491353-13614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175691491353-13614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175691491353-13614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260923 15:02:11.983502 22489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:11.974708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:40153 (local address 127.13.75.129:45829)
0923 15:02:11.974944 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0923 15:02:11.974994 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0923 15:02:11.975530 (+ 536us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:02:11.975973 (+ 443us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:02:11.975977 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:02:11.976613 (+ 636us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:02:11.976815 (+ 202us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:11.976821 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:11.978044 (+ 1223us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:11.978045 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:11.979032 (+ 987us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:11.979035 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:11.979654 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:11.979656 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:11.979772 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:11.980308 (+ 536us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:02:11.980317 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:02:11.983215 (+ 2898us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.75.190:40153: Server oryx/127.13.75.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":100,"threads_started":1}
W20260923 15:02:11.983781 22487 heartbeater.cc:644] Failed to heartbeat to 127.13.75.190:40153 (0 consecutive failures): Not authorized: Failed to ping master at 127.13.75.190:40153: Client connection negotiation failed: client connection to 127.13.75.190:40153: Server oryx/127.13.75.190@KRBTEST.COM not found in Kerberos database
W20260923 15:02:11.984381 22490 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 15:02:11.974759 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.129:45829 (local address 127.13.75.190:40153)
0923 15:02:11.974988 (+ 229us) server_negotiation.cc:206] Beginning negotiation
0923 15:02:11.974999 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0923 15:02:11.975096 (+ 97us) server_negotiation.cc:407] Connection header received
0923 15:02:11.975732 (+ 636us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:02:11.975739 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:02:11.975788 (+ 49us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:02:11.975911 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:02:11.976974 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:11.977905 (+ 931us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:11.979121 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:11.979542 (+ 421us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:11.979628 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:11.984167 (+ 4539us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.75.129:45829: BlockingRecv error: recv got EOF from 127.13.75.129:45829 (error 108)
Metrics: {"server-negotiator.queue_time_us":170,"thread_start_us":139,"threads_started":1}
Sep 23 15:02:11 dist-test-slave-cnb0 krb5kdc[21755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.75.129@KRBTEST.COM for oryx/127.13.75.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 15:02:11 dist-test-slave-cnb0 krb5kdc[21755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.75.129@KRBTEST.COM for oryx/127.13.75.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 15:02:11.989434 22489 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:11.984061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:40153 (local address 127.13.75.129:55869)
0923 15:02:11.984120 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0923 15:02:11.984133 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0923 15:02:11.984185 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:02:11.984880 (+ 695us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:02:11.984881 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:02:11.985071 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:02:11.985139 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:11.985141 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:11.985846 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:11.985848 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:11.986452 (+ 604us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:11.986454 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:11.987069 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:11.987070 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:11.987126 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:11.987655 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:02:11.987663 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:02:11.989378 (+ 1715us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.75.190:40153: Server oryx/127.13.75.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":38}
W20260923 15:02:11.989619 22490 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 15:02:11.984643 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.129:55869 (local address 127.13.75.190:40153)
0923 15:02:11.984708 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0923 15:02:11.984711 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:02:11.984722 (+ 11us) server_negotiation.cc:407] Connection header received
0923 15:02:11.984754 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:02:11.984755 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:02:11.984774 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:02:11.984854 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:02:11.985247 (+ 393us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:11.985759 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:11.986561 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:11.986944 (+ 383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:11.987057 (+ 113us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:11.989524 (+ 2467us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.75.129:55869: BlockingRecv error: recv got EOF from 127.13.75.129:55869 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
Sep 23 15:02:11 dist-test-slave-cnb0 krb5kdc[21755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.75.129@KRBTEST.COM for oryx/127.13.75.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 15:02:11 dist-test-slave-cnb0 krb5kdc[21755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.75.129@KRBTEST.COM for oryx/127.13.75.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 15:02:11.994784 22492 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:11.989704 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:40153 (local address 127.13.75.129:45705)
0923 15:02:11.989865 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0923 15:02:11.989876 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 15:02:11.989928 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:02:11.990105 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:02:11.990105 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:02:11.990242 (+ 137us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:02:11.990293 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:11.990294 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:11.991319 (+ 1025us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:11.991320 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:11.991888 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:11.991890 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:11.992430 (+ 540us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:11.992431 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:11.992479 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:11.992996 (+ 517us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:02:11.993001 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:02:11.994716 (+ 1715us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.75.190:40153: Server oryx/127.13.75.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":89,"threads_started":1}
W20260923 15:02:11.994946 22490 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 15:02:11.989892 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.129:45705 (local address 127.13.75.190:40153)
0923 15:02:11.989958 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0923 15:02:11.989960 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:02:11.989972 (+ 12us) server_negotiation.cc:407] Connection header received
0923 15:02:11.990005 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:02:11.990006 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:02:11.990021 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:02:11.990095 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:02:11.990369 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:11.991236 (+ 867us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:11.991971 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:11.992328 (+ 357us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:11.992411 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:11.994841 (+ 2430us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.75.129:45705: BlockingRecv error: recv got EOF from 127.13.75.129:45705 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
W20260923 15:02:11.995039 22487 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260923 15:02:12.071027 22491 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:02:12.071269 22491 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:02:12.071300 22491 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:02:12.072508 22491 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:02:12.072546 22491 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:02:12.072575 22491 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.75.130
I20260923 15:02:12.074010 22491 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.75.148:35183
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175691491353-13614-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175691491353-13614-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175691491353-13614-0/krb5kdc/oryx_127.13.75.130.keytab
--principal=oryx/127.13.75.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.75.130:33961
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175691491353-13614-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.75.130
--webserver_port=38093
--tserver_master_addrs=127.13.75.190:40153
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22491
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.75.130
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175691491353-13614-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:02:12.074209 22491 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:02:12.074406 22491 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:02:12.079828 22498 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:02:12.079851 22500 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:02:12.079851 22497 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:02:12.080298 22491 server_base.cc:1034] running on GCE node
I20260923 15:02:12.080616 22491 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:02:12.080892 22491 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:02:12.082064 22491 hybrid_clock.cc:648] HybridClock initialized: now 1790175732082042 us; error 37 us; skew 500 ppm
Sep 23 15:02:12 dist-test-slave-cnb0 krb5kdc[21755](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175732, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.75.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:02:12.085052 22491 init.cc:377] Logged in from keytab as oryx/127.13.75.130@KRBTEST.COM (short username oryx)
I20260923 15:02:12.085747 22491 webserver.cc:460] Webserver started at http://127.13.75.130:38093/ using document root <none> and password file <none>
I20260923 15:02:12.086338 22491 fs_manager.cc:359] Metadata directory not provided
I20260923 15:02:12.086499 22491 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:02:12.088559 22491 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:02:12.089535 22507 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:02:12.089869 22491 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:02:12.090018 22491 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175691491353-13614-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175691491353-13614-0/minicluster-data/ts-1/wal
uuid: "13e655804acf4f37a894d688026ed923"
format_stamp: "Formatted at 2026-09-23 15:02:11 on dist-test-slave-cnb0"
server_key: "23233fa036f2316b56ee108d45140e29"
server_key_iv: "ac9d889d05de5db6fd6b56af408628e0"
server_key_version: "encryptionkey@0"
I20260923 15:02:12.090417 22491 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175691491353-13614-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175691491353-13614-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175691491353-13614-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:02:12.108496 22491 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:02:12.111189 22491 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:02:12.111375 22491 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:02:12.111650 22491 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 15:02:12.111996 22491 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 15:02:12.112054 22491 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 15:02:12.112089 22491 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 15:02:12.112118 22491 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 15:02:12.118196 22491 rpc_server.cc:307] RPC server started. Bound to: 127.13.75.130:33961
I20260923 15:02:12.118239 22620 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.75.130:33961 every 8 connection(s)
I20260923 15:02:12.118604 22491 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175691491353-13614-0/minicluster-data/ts-1/data/info.pb
I20260923 15:02:12.120153 13614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKC4soy/build/release/bin/kudu as pid 22491
I20260923 15:02:12.120481 13614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKC4soy/build/release/bin/kudu
/tmp/dist-test-taskKC4soy/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175691491353-13614-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175691491353-13614-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175691491353-13614-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175691491353-13614-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.75.131:35399
--local_ip_for_outbound_sockets=127.13.75.131
--tserver_master_addrs=127.13.75.190:40153
--webserver_port=45091
--webserver_interface=127.13.75.131
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.131.keytab
--principal=kudu/127.13.75.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.75.148:35183
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--principal=oryx/127.13.75.131
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175691491353-13614-0/krb5kdc/oryx_127.13.75.131.keytab with env {KRB5CCNAME=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175691491353-13614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175691491353-13614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175691491353-13614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 23 15:02:12 dist-test-slave-cnb0 krb5kdc[21755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.75.130@KRBTEST.COM for oryx/127.13.75.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 15:02:12 dist-test-slave-cnb0 krb5kdc[21755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.75.130@KRBTEST.COM for oryx/127.13.75.190@KRBTEST.COM, Server not found in Kerberos database
W20260923 15:02:12.128146 22490 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 15:02:12.119485 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.130:51851 (local address 127.13.75.190:40153)
0923 15:02:12.119665 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0923 15:02:12.119668 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:02:12.119849 (+ 181us) server_negotiation.cc:407] Connection header received
0923 15:02:12.120672 (+ 823us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:02:12.120674 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:02:12.120693 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:02:12.120724 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:02:12.122282 (+ 1558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:12.122789 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:12.123929 (+ 1140us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:12.124141 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:12.124235 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:12.128092 (+ 3857us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.75.130:51851: BlockingRecv error: recv got EOF from 127.13.75.130:51851 (error 108)
Metrics: {"server-negotiator.queue_time_us":137}
I20260923 15:02:12.128273 22623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:12.119375 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:40153 (local address 127.13.75.130:51851)
0923 15:02:12.119699 (+ 324us) negotiation.cc:107] Waiting for socket to connect
0923 15:02:12.119760 (+ 61us) client_negotiation.cc:174] Beginning negotiation
0923 15:02:12.120497 (+ 737us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:02:12.120931 (+ 434us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:02:12.120936 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:02:12.121822 (+ 886us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:02:12.122154 (+ 332us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:12.122161 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:12.122933 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:12.122935 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:12.123795 (+ 860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:12.123798 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:12.124250 (+ 452us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:12.124251 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:12.124392 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:12.125039 (+ 647us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:02:12.125059 (+ 20us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:02:12.127968 (+ 2909us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.75.190:40153: Server oryx/127.13.75.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":276,"thread_start_us":119,"threads_started":1}
W20260923 15:02:12.128593 22621 heartbeater.cc:644] Failed to heartbeat to 127.13.75.190:40153 (0 consecutive failures): Not authorized: Failed to ping master at 127.13.75.190:40153: Client connection negotiation failed: client connection to 127.13.75.190:40153: Server oryx/127.13.75.190@KRBTEST.COM not found in Kerberos database
Sep 23 15:02:12 dist-test-slave-cnb0 krb5kdc[21755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.75.130@KRBTEST.COM for oryx/127.13.75.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 15:02:12 dist-test-slave-cnb0 krb5kdc[21755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.75.130@KRBTEST.COM for oryx/127.13.75.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 15:02:12.134723 22623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:12.128891 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:40153 (local address 127.13.75.130:41695)
0923 15:02:12.129024 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0923 15:02:12.129039 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0923 15:02:12.129114 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:02:12.129278 (+ 164us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:02:12.129279 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:02:12.129478 (+ 199us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:02:12.129564 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:12.129566 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:12.130274 (+ 708us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:12.130275 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:12.131404 (+ 1129us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:12.131408 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:12.131856 (+ 448us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:12.131858 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:12.131948 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:12.132492 (+ 544us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:02:12.132499 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:02:12.134627 (+ 2128us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.75.190:40153: Server oryx/127.13.75.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":111}
W20260923 15:02:12.134784 22490 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 15:02:12.128968 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.130:41695 (local address 127.13.75.190:40153)
0923 15:02:12.129016 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0923 15:02:12.129018 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:02:12.129092 (+ 74us) server_negotiation.cc:407] Connection header received
0923 15:02:12.129192 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:02:12.129193 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:02:12.129202 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:02:12.129250 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:02:12.129649 (+ 399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:12.130188 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:12.131519 (+ 1331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:12.131760 (+ 241us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:12.131836 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:12.134720 (+ 2884us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.75.130:41695: BlockingRecv error: recv got EOF from 127.13.75.130:41695 (error 108)
Metrics: {"server-negotiator.queue_time_us":21}
Sep 23 15:02:12 dist-test-slave-cnb0 krb5kdc[21755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.75.130@KRBTEST.COM for oryx/127.13.75.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 15:02:12 dist-test-slave-cnb0 krb5kdc[21755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.75.130@KRBTEST.COM for oryx/127.13.75.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 15:02:12.139991 22623 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:12.135061 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:40153 (local address 127.13.75.130:52459)
0923 15:02:12.135268 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0923 15:02:12.135277 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 15:02:12.135340 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:02:12.135480 (+ 140us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:02:12.135481 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:02:12.135621 (+ 140us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:02:12.135656 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:12.135658 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:12.136332 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:12.136333 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:12.136944 (+ 611us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:12.136946 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:12.137343 (+ 397us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:12.137344 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:12.137396 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:12.137914 (+ 518us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:02:12.137919 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:02:12.139903 (+ 1984us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.75.190:40153: Server oryx/127.13.75.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":195}
W20260923 15:02:12.140076 22490 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 15:02:12.135111 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.130:52459 (local address 127.13.75.190:40153)
0923 15:02:12.135171 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0923 15:02:12.135173 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:02:12.135324 (+ 151us) server_negotiation.cc:407] Connection header received
0923 15:02:12.135393 (+ 69us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:02:12.135394 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:02:12.135404 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:02:12.135476 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:02:12.135744 (+ 268us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:12.136246 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:12.137048 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:12.137257 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:12.137342 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:12.140004 (+ 2662us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.75.130:52459: BlockingRecv error: recv got EOF from 127.13.75.130:52459 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
W20260923 15:02:12.140334 22621 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260923 15:02:12.211658 22624 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:02:12.211928 22624 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:02:12.211973 22624 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:02:12.213268 22624 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:02:12.213349 22624 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:02:12.213415 22624 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.75.131
I20260923 15:02:12.215070 22624 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.75.148:35183
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175691491353-13614-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175691491353-13614-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175691491353-13614-0/krb5kdc/oryx_127.13.75.131.keytab
--principal=oryx/127.13.75.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.75.131:35399
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175691491353-13614-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.75.131
--webserver_port=45091
--tserver_master_addrs=127.13.75.190:40153
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22624
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.75.131
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175691491353-13614-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:02:12.215375 22624 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:02:12.215641 22624 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:02:12.221248 22632 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:02:12.221364 22630 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:02:12.221411 22629 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:02:12.221660 22624 server_base.cc:1034] running on GCE node
I20260923 15:02:12.221892 22624 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:02:12.222150 22624 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:02:12.223321 22624 hybrid_clock.cc:648] HybridClock initialized: now 1790175732223303 us; error 49 us; skew 500 ppm
Sep 23 15:02:12 dist-test-slave-cnb0 krb5kdc[21755](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175732, etypes {rep=17 tkt=17 ses=17}, oryx/127.13.75.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:02:12.226549 22624 init.cc:377] Logged in from keytab as oryx/127.13.75.131@KRBTEST.COM (short username oryx)
I20260923 15:02:12.227214 22624 webserver.cc:460] Webserver started at http://127.13.75.131:45091/ using document root <none> and password file <none>
I20260923 15:02:12.227603 22624 fs_manager.cc:359] Metadata directory not provided
I20260923 15:02:12.227710 22624 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:02:12.229305 22624 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 15:02:12.230062 22639 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:02:12.230269 22624 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:02:12.230386 22624 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175691491353-13614-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175691491353-13614-0/minicluster-data/ts-2/wal
uuid: "4d63b5f9ed9d477bb24b09f7498aebaa"
format_stamp: "Formatted at 2026-09-23 15:02:11 on dist-test-slave-cnb0"
server_key: "a954a76a88a427cf434eef5ef8704e5c"
server_key_iv: "58e9ab407c098fbb240a5ad4e380c3a9"
server_key_version: "encryptionkey@0"
I20260923 15:02:12.230758 22624 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175691491353-13614-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175691491353-13614-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175691491353-13614-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:02:12.248462 22624 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:02:12.251212 22624 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:02:12.251441 22624 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:02:12.251730 22624 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 15:02:12.252103 22624 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 15:02:12.252164 22624 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 15:02:12.252205 22624 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 15:02:12.252259 22624 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 15:02:12.258692 22624 rpc_server.cc:307] RPC server started. Bound to: 127.13.75.131:35399
I20260923 15:02:12.258733 22752 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.75.131:35399 every 8 connection(s)
I20260923 15:02:12.259217 22624 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestMismatchingPrincipals.1790175691491353-13614-0/minicluster-data/ts-2/data/info.pb
Sep 23 15:02:12 dist-test-slave-cnb0 krb5kdc[21755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.75.131@KRBTEST.COM for oryx/127.13.75.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 15:02:12.266593 13614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKC4soy/build/release/bin/kudu as pid 22624
Sep 23 15:02:12 dist-test-slave-cnb0 krb5kdc[21755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.75.131@KRBTEST.COM for oryx/127.13.75.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 15:02:12.268093 22755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:12.260080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:40153 (local address 127.13.75.131:45041)
0923 15:02:12.260266 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0923 15:02:12.260314 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0923 15:02:12.260881 (+ 567us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:02:12.261166 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:02:12.261170 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:02:12.261873 (+ 703us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:02:12.262096 (+ 223us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:12.262102 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:12.262843 (+ 741us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:12.262845 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:12.263909 (+ 1064us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:12.263912 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:12.264337 (+ 425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:12.264338 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:12.264503 (+ 165us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:12.265154 (+ 651us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:02:12.265164 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:02:12.267916 (+ 2752us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.75.190:40153: Server oryx/127.13.75.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":102,"threads_started":1}
W20260923 15:02:12.268119 22490 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 15:02:12.260118 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.131:45041 (local address 127.13.75.190:40153)
0923 15:02:12.260168 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0923 15:02:12.260173 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 15:02:12.260443 (+ 270us) server_negotiation.cc:407] Connection header received
0923 15:02:12.260992 (+ 549us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:02:12.260994 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:02:12.261005 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:02:12.261086 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:02:12.262237 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:12.262737 (+ 500us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:12.264026 (+ 1289us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:12.264239 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:12.264315 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:12.268029 (+ 3714us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.75.131:45041: BlockingRecv error: recv got EOF from 127.13.75.131:45041 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260923 15:02:12.268389 22753 heartbeater.cc:644] Failed to heartbeat to 127.13.75.190:40153 (0 consecutive failures): Not authorized: Failed to ping master at 127.13.75.190:40153: Client connection negotiation failed: client connection to 127.13.75.190:40153: Server oryx/127.13.75.190@KRBTEST.COM not found in Kerberos database
Sep 23 15:02:12 dist-test-slave-cnb0 krb5kdc[21755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.75.131@KRBTEST.COM for oryx/127.13.75.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 15:02:12 dist-test-slave-cnb0 krb5kdc[21755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.75.131@KRBTEST.COM for oryx/127.13.75.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 15:02:12.273790 22755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:12.268629 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:40153 (local address 127.13.75.131:47785)
0923 15:02:12.268759 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0923 15:02:12.268769 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 15:02:12.268818 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:02:12.268971 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:02:12.268972 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:02:12.269095 (+ 123us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:02:12.269145 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:12.269147 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:12.269784 (+ 637us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:12.269785 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:12.270491 (+ 706us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:12.270493 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:12.271124 (+ 631us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:12.271126 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:12.271198 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:12.271600 (+ 402us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:02:12.271607 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:02:12.273733 (+ 2126us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.75.190:40153: Server oryx/127.13.75.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":105}
W20260923 15:02:12.273919 22490 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 15:02:12.268705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.131:47785 (local address 127.13.75.190:40153)
0923 15:02:12.268774 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0923 15:02:12.268776 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:02:12.268804 (+ 28us) server_negotiation.cc:407] Connection header received
0923 15:02:12.268870 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:02:12.268871 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:02:12.268882 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:02:12.268931 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:02:12.269226 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:12.269697 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:12.270659 (+ 962us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:12.271026 (+ 367us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:12.271112 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:12.273858 (+ 2746us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.75.131:47785: BlockingRecv error: recv got EOF from 127.13.75.131:47785 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
I20260923 15:02:12.276455 22757 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:12.267256 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48112 (local address 127.13.75.190:40153)
0923 15:02:12.267385 (+ 129us) server_negotiation.cc:206] Beginning negotiation
0923 15:02:12.267387 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:02:12.267499 (+ 112us) server_negotiation.cc:407] Connection header received
0923 15:02:12.267589 (+ 90us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:02:12.267590 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:02:12.267604 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:02:12.267693 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:02:12.268468 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:12.269179 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:12.270083 (+ 904us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:12.270272 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:12.270323 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:12.272048 (+ 1725us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:02:12.272054 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:02:12.272054 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:02:12.272067 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:02:12.274398 (+ 2331us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:02:12.274796 (+ 398us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:02:12.274800 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:02:12.274800 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:02:12.274971 (+ 171us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:02:12.275259 (+ 288us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:02:12.275260 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:02:12.275260 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:02:12.275609 (+ 349us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:02:12.275717 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0923 15:02:12.276206 (+ 489us) server_negotiation.cc:299] Negotiation successful
0923 15:02:12.276337 (+ 131us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":104,"thread_start_us":76,"threads_started":1}
Sep 23 15:02:12 dist-test-slave-cnb0 krb5kdc[21755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.75.131@KRBTEST.COM for oryx/127.13.75.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 15:02:12 dist-test-slave-cnb0 krb5kdc[21755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.75.131@KRBTEST.COM for oryx/127.13.75.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 15:02:12.279598 22755 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:12.274102 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:40153 (local address 127.13.75.131:40633)
0923 15:02:12.274202 (+ 100us) negotiation.cc:107] Waiting for socket to connect
0923 15:02:12.274214 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 15:02:12.274266 (+ 52us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:02:12.274400 (+ 134us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:02:12.274400 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:02:12.274562 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:02:12.274606 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:12.274607 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:12.275494 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:12.275495 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:12.276269 (+ 774us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:12.276272 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:12.276864 (+ 592us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:12.276865 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:12.276933 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:12.277491 (+ 558us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:02:12.277497 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:02:12.279475 (+ 1978us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.75.190:40153: Server oryx/127.13.75.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":80}
W20260923 15:02:12.279613 22490 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 15:02:12.274153 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.131:40633 (local address 127.13.75.190:40153)
0923 15:02:12.274212 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0923 15:02:12.274214 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:02:12.274248 (+ 34us) server_negotiation.cc:407] Connection header received
0923 15:02:12.274327 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:02:12.274328 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:02:12.274339 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:02:12.274393 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:02:12.274687 (+ 294us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:12.275386 (+ 699us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:12.276387 (+ 1001us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:12.276767 (+ 380us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:12.276870 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:12.279559 (+ 2689us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.75.131:40633: BlockingRecv error: recv got EOF from 127.13.75.131:40633 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
W20260923 15:02:12.279812 22753 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
Sep 23 15:02:12 dist-test-slave-cnb0 krb5kdc[21755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.75.129@KRBTEST.COM for oryx/127.13.75.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 15:02:12 dist-test-slave-cnb0 krb5kdc[21755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.75.129@KRBTEST.COM for oryx/127.13.75.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 15:02:13.001928 22758 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:12.995608 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:40153 (local address 127.13.75.129:54285)
0923 15:02:12.995757 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0923 15:02:12.995777 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0923 15:02:12.995831 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:02:12.996222 (+ 391us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:02:12.996224 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:02:12.996417 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:02:12.996516 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:12.996519 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:12.997290 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:12.997292 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:12.997928 (+ 636us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:12.997931 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:12.998473 (+ 542us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:12.998474 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:12.998566 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:12.999470 (+ 904us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:02:12.999475 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:02:13.001859 (+ 2384us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.75.190:40153: Server oryx/127.13.75.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":92,"threads_started":1}
W20260923 15:02:13.002175 22759 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 15:02:12.995739 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.129:54285 (local address 127.13.75.190:40153)
0923 15:02:12.995902 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0923 15:02:12.995904 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:02:12.995915 (+ 11us) server_negotiation.cc:407] Connection header received
0923 15:02:12.995950 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:02:12.995951 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:02:12.996039 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:02:12.996164 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:02:12.996633 (+ 469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:12.997173 (+ 540us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:12.998061 (+ 888us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:12.998294 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:12.998371 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:13.002074 (+ 3703us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.75.129:54285: BlockingRecv error: recv got EOF from 127.13.75.129:54285 (error 108)
Metrics: {"server-negotiator.queue_time_us":133,"thread_start_us":102,"threads_started":1}
Sep 23 15:02:13 dist-test-slave-cnb0 krb5kdc[21755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.75.130@KRBTEST.COM for oryx/127.13.75.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 15:02:13 dist-test-slave-cnb0 krb5kdc[21755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.75.130@KRBTEST.COM for oryx/127.13.75.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 15:02:13.146353 22760 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:13.140767 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:40153 (local address 127.13.75.130:47957)
0923 15:02:13.140974 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0923 15:02:13.140990 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0923 15:02:13.141090 (+ 100us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:02:13.141268 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:02:13.141269 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:02:13.141469 (+ 200us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:02:13.141530 (+ 61us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:13.141532 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:13.142308 (+ 776us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:13.142309 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:13.143029 (+ 720us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:13.143032 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:13.143494 (+ 462us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:13.143496 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:13.143564 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:13.144091 (+ 527us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:02:13.144097 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:02:13.146228 (+ 2131us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.75.190:40153: Server oryx/127.13.75.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":98,"threads_started":1}
W20260923 15:02:13.146405 22759 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 15:02:13.140925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.130:47957 (local address 127.13.75.190:40153)
0923 15:02:13.141002 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0923 15:02:13.141004 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:02:13.141069 (+ 65us) server_negotiation.cc:407] Connection header received
0923 15:02:13.141163 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:02:13.141164 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:02:13.141179 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:02:13.141260 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:02:13.141642 (+ 382us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:13.142201 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:13.143131 (+ 930us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:13.143360 (+ 229us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:13.143481 (+ 121us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:13.146330 (+ 2849us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.75.130:47957: BlockingRecv error: recv got EOF from 127.13.75.130:47957 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Sep 23 15:02:13 dist-test-slave-cnb0 krb5kdc[21755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.75.131@KRBTEST.COM for oryx/127.13.75.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 15:02:13 dist-test-slave-cnb0 krb5kdc[21755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.75.131@KRBTEST.COM for oryx/127.13.75.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 15:02:13.286195 22761 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:13.280319 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:40153 (local address 127.13.75.131:56183)
0923 15:02:13.280560 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0923 15:02:13.280571 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 15:02:13.280624 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:02:13.280782 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:02:13.280782 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:02:13.280987 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:02:13.281038 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:13.281040 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:13.281793 (+ 753us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:13.281795 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:13.282384 (+ 589us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:13.282386 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:13.283035 (+ 649us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:13.283037 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:13.283086 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:13.283678 (+ 592us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:02:13.283683 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:02:13.286086 (+ 2403us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.75.190:40153: Server oryx/127.13.75.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":212,"thread_start_us":109,"threads_started":1}
W20260923 15:02:13.286324 22759 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 15:02:13.280475 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.131:56183 (local address 127.13.75.190:40153)
0923 15:02:13.280612 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0923 15:02:13.280615 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:02:13.280628 (+ 13us) server_negotiation.cc:407] Connection header received
0923 15:02:13.280676 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:02:13.280678 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:02:13.280692 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:02:13.280759 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:02:13.281178 (+ 419us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:13.281688 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:13.282506 (+ 818us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:13.282913 (+ 407us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:13.282999 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:13.286235 (+ 3236us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.75.131:56183: BlockingRecv error: recv got EOF from 127.13.75.131:56183 (error 108)
Metrics: {"server-negotiator.queue_time_us":100}
Sep 23 15:02:14 dist-test-slave-cnb0 krb5kdc[21755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.75.129@KRBTEST.COM for oryx/127.13.75.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 15:02:14 dist-test-slave-cnb0 krb5kdc[21755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.75.129@KRBTEST.COM for oryx/127.13.75.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 15:02:14.008643 22762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:14.002672 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:40153 (local address 127.13.75.129:45999)
0923 15:02:14.002843 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0923 15:02:14.002856 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0923 15:02:14.003009 (+ 153us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:02:14.003228 (+ 219us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:02:14.003230 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:02:14.003403 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:02:14.003460 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:14.003462 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:14.004451 (+ 989us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:14.004452 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:14.005052 (+ 600us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:14.005054 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:14.005485 (+ 431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:14.005486 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:14.005539 (+ 53us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:14.006087 (+ 548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:02:14.006093 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:02:14.008536 (+ 2443us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.75.190:40153: Server oryx/127.13.75.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":116,"threads_started":1}
W20260923 15:02:14.008718 22763 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 15:02:14.002864 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.129:45999 (local address 127.13.75.190:40153)
0923 15:02:14.003033 (+ 169us) server_negotiation.cc:206] Beginning negotiation
0923 15:02:14.003035 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:02:14.003044 (+ 9us) server_negotiation.cc:407] Connection header received
0923 15:02:14.003072 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:02:14.003073 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:02:14.003083 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:02:14.003162 (+ 79us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:02:14.003619 (+ 457us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:14.004315 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:14.005165 (+ 850us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:14.005393 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:14.005465 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:14.008644 (+ 3179us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.75.129:45999: BlockingRecv error: recv got EOF from 127.13.75.129:45999 (error 108)
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":93,"threads_started":1}
Sep 23 15:02:14 dist-test-slave-cnb0 krb5kdc[21755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.75.130@KRBTEST.COM for oryx/127.13.75.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 15:02:14 dist-test-slave-cnb0 krb5kdc[21755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.75.130@KRBTEST.COM for oryx/127.13.75.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 15:02:14.152809 22764 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:14.147235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:40153 (local address 127.13.75.130:47629)
0923 15:02:14.147464 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0923 15:02:14.147478 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0923 15:02:14.147586 (+ 108us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:02:14.147755 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:02:14.147756 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:02:14.147961 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:02:14.148032 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:14.148034 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:14.148784 (+ 750us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:14.148786 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:14.149382 (+ 596us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:14.149383 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:14.149816 (+ 433us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:14.149817 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:14.149884 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:14.150464 (+ 580us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:02:14.150469 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:02:14.152701 (+ 2232us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.75.190:40153: Server oryx/127.13.75.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":209,"thread_start_us":110,"threads_started":1}
W20260923 15:02:14.152882 22763 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 15:02:14.147333 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.130:47629 (local address 127.13.75.190:40153)
0923 15:02:14.147457 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0923 15:02:14.147459 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:02:14.147568 (+ 109us) server_negotiation.cc:407] Connection header received
0923 15:02:14.147655 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:02:14.147657 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:02:14.147671 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:02:14.147752 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:02:14.148129 (+ 377us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:14.148687 (+ 558us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:14.149489 (+ 802us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:14.149717 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:14.149816 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:14.152810 (+ 2994us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.75.130:47629: BlockingRecv error: recv got EOF from 127.13.75.130:47629 (error 108)
Metrics: {"server-negotiator.queue_time_us":82}
Sep 23 15:02:14 dist-test-slave-cnb0 krb5kdc[21755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.75.131@KRBTEST.COM for oryx/127.13.75.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 15:02:14 dist-test-slave-cnb0 krb5kdc[21755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.75.131@KRBTEST.COM for oryx/127.13.75.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 15:02:14.292320 22765 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:14.286820 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:40153 (local address 127.13.75.131:35401)
0923 15:02:14.287010 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0923 15:02:14.287026 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0923 15:02:14.287166 (+ 140us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:02:14.287373 (+ 207us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:02:14.287375 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:02:14.287552 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:02:14.287601 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:14.287604 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:14.288333 (+ 729us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:14.288334 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:14.288860 (+ 526us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:14.288862 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:14.289271 (+ 409us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:14.289272 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:14.289323 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:14.289908 (+ 585us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:02:14.289914 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:02:14.292257 (+ 2343us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.75.190:40153: Server oryx/127.13.75.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":103,"threads_started":1}
W20260923 15:02:14.292475 22763 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 15:02:14.287079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.131:35401 (local address 127.13.75.190:40153)
0923 15:02:14.287148 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0923 15:02:14.287150 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:02:14.287161 (+ 11us) server_negotiation.cc:407] Connection header received
0923 15:02:14.287242 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:02:14.287243 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:02:14.287264 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:02:14.287321 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:02:14.287712 (+ 391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:14.288248 (+ 536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:14.288945 (+ 697us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:14.289167 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:14.289227 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:14.292373 (+ 3146us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.75.131:35401: BlockingRecv error: recv got EOF from 127.13.75.131:35401 (error 108)
Metrics: {"server-negotiator.queue_time_us":43}
Sep 23 15:02:15 dist-test-slave-cnb0 krb5kdc[21755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.75.129@KRBTEST.COM for oryx/127.13.75.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 15:02:15 dist-test-slave-cnb0 krb5kdc[21755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.75.129@KRBTEST.COM for oryx/127.13.75.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 15:02:15.015049 22766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:15.009224 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:40153 (local address 127.13.75.129:37921)
0923 15:02:15.009377 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0923 15:02:15.009397 (+ 20us) client_negotiation.cc:174] Beginning negotiation
0923 15:02:15.009450 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:02:15.009792 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:02:15.009794 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:02:15.009972 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:02:15.010032 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:15.010034 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:15.010799 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:15.010800 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:15.011483 (+ 683us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:15.011486 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:15.011939 (+ 453us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:15.011940 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:15.012007 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:15.012634 (+ 627us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:02:15.012640 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:02:15.014973 (+ 2333us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.75.190:40153: Server oryx/127.13.75.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":98,"threads_started":1}
W20260923 15:02:15.015208 22767 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 15:02:15.009443 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.129:37921 (local address 127.13.75.190:40153)
0923 15:02:15.009603 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0923 15:02:15.009605 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:02:15.009614 (+ 9us) server_negotiation.cc:407] Connection header received
0923 15:02:15.009638 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:02:15.009639 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:02:15.009650 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:02:15.009722 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:02:15.010129 (+ 407us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:15.010701 (+ 572us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:15.011604 (+ 903us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:15.011839 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:15.011913 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:15.015106 (+ 3193us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.75.129:37921: BlockingRecv error: recv got EOF from 127.13.75.129:37921 (error 108)
Metrics: {"server-negotiator.queue_time_us":140,"thread_start_us":62,"threads_started":1}
Sep 23 15:02:15 dist-test-slave-cnb0 krb5kdc[21755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.75.130@KRBTEST.COM for oryx/127.13.75.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 15:02:15 dist-test-slave-cnb0 krb5kdc[21755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.75.130@KRBTEST.COM for oryx/127.13.75.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 15:02:15.158811 22768 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:15.153474 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:40153 (local address 127.13.75.130:49703)
0923 15:02:15.153640 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0923 15:02:15.153654 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0923 15:02:15.153735 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:02:15.153925 (+ 190us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:02:15.153926 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:02:15.154094 (+ 168us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:02:15.154152 (+ 58us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:15.154154 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:15.155054 (+ 900us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:15.155056 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:15.155994 (+ 938us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:15.155998 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:15.156361 (+ 363us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:15.156362 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:15.156406 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:15.156956 (+ 550us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:02:15.156961 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:02:15.158752 (+ 1791us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.75.190:40153: Server oryx/127.13.75.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":77,"threads_started":1}
W20260923 15:02:15.159039 22767 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 15:02:15.153600 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.130:49703 (local address 127.13.75.190:40153)
0923 15:02:15.153716 (+ 116us) server_negotiation.cc:206] Beginning negotiation
0923 15:02:15.153718 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:02:15.153732 (+ 14us) server_negotiation.cc:407] Connection header received
0923 15:02:15.153795 (+ 63us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:02:15.153796 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:02:15.153809 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:02:15.153889 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:02:15.154287 (+ 398us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:15.154867 (+ 580us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:15.156083 (+ 1216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:15.156285 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:15.156350 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:15.158953 (+ 2603us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.75.130:49703: BlockingRecv error: recv got EOF from 127.13.75.130:49703 (error 108)
Metrics: {"server-negotiator.queue_time_us":81}
Sep 23 15:02:15 dist-test-slave-cnb0 krb5kdc[21755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.75.131@KRBTEST.COM for oryx/127.13.75.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 15:02:15 dist-test-slave-cnb0 krb5kdc[21755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.75.131@KRBTEST.COM for oryx/127.13.75.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 15:02:15.297848 22769 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:15.292860 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:40153 (local address 127.13.75.131:48729)
0923 15:02:15.293013 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0923 15:02:15.293023 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 15:02:15.293077 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:02:15.293214 (+ 137us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:02:15.293215 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:02:15.293373 (+ 158us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:02:15.293424 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:15.293426 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:15.294180 (+ 754us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:15.294181 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:15.294729 (+ 548us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:15.294731 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:15.295237 (+ 506us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:15.295239 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:15.295285 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:15.295811 (+ 526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:02:15.295816 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:02:15.297787 (+ 1971us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.75.190:40153: Server oryx/127.13.75.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":98,"threads_started":1}
W20260923 15:02:15.298018 22767 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 15:02:15.292911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.131:48729 (local address 127.13.75.190:40153)
0923 15:02:15.292979 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0923 15:02:15.292982 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:02:15.293081 (+ 99us) server_negotiation.cc:407] Connection header received
0923 15:02:15.293124 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:02:15.293124 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:02:15.293137 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:02:15.293199 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:02:15.293489 (+ 290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:15.294109 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:15.294859 (+ 750us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:15.295141 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:15.295205 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:15.297923 (+ 2718us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.75.131:48729: BlockingRecv error: recv got EOF from 127.13.75.131:48729 (error 108)
Metrics: {"server-negotiator.queue_time_us":41}
Sep 23 15:02:16 dist-test-slave-cnb0 krb5kdc[21755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.75.129@KRBTEST.COM for oryx/127.13.75.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 15:02:16 dist-test-slave-cnb0 krb5kdc[21755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.75.129@KRBTEST.COM for oryx/127.13.75.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 15:02:16.026054 22770 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:16.015687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:40153 (local address 127.13.75.129:44823)
0923 15:02:16.015833 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0923 15:02:16.015843 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 15:02:16.015894 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:02:16.016038 (+ 144us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:02:16.016039 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:02:16.016205 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:02:16.016257 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:16.016259 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:16.016964 (+ 705us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:16.016965 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:16.017518 (+ 553us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:16.017519 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:16.017878 (+ 359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:16.017879 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:16.017924 (+ 45us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:16.023681 (+ 5757us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:02:16.023687 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:02:16.025968 (+ 2281us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.75.190:40153: Server oryx/127.13.75.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":86,"threads_started":1}
W20260923 15:02:16.026191 22771 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 15:02:16.015767 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.129:44823 (local address 127.13.75.190:40153)
0923 15:02:16.015911 (+ 144us) server_negotiation.cc:206] Beginning negotiation
0923 15:02:16.015913 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:02:16.015921 (+ 8us) server_negotiation.cc:407] Connection header received
0923 15:02:16.015944 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:02:16.015944 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:02:16.015955 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:02:16.016022 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:02:16.016325 (+ 303us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:16.016872 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:16.017607 (+ 735us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:16.017805 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:16.017862 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:16.026080 (+ 8218us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.75.129:44823: BlockingRecv error: recv got EOF from 127.13.75.129:44823 (error 108)
Metrics: {"server-negotiator.queue_time_us":118,"thread_start_us":83,"threads_started":1}
Sep 23 15:02:16 dist-test-slave-cnb0 krb5kdc[21755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.75.130@KRBTEST.COM for oryx/127.13.75.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 15:02:16 dist-test-slave-cnb0 krb5kdc[21755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.75.130@KRBTEST.COM for oryx/127.13.75.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 15:02:16.164529 22772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:16.159436 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:40153 (local address 127.13.75.130:51375)
0923 15:02:16.159676 (+ 240us) negotiation.cc:107] Waiting for socket to connect
0923 15:02:16.159689 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0923 15:02:16.159784 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:02:16.159969 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:02:16.159971 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:02:16.160157 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:02:16.160206 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:16.160208 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:16.160911 (+ 703us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:16.160912 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:16.161444 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:16.161446 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:16.161828 (+ 382us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:16.161829 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:16.161888 (+ 59us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:16.162393 (+ 505us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:02:16.162398 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:02:16.164443 (+ 2045us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.75.190:40153: Server oryx/127.13.75.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":219,"thread_start_us":111,"threads_started":1}
W20260923 15:02:16.164698 22771 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 15:02:16.159571 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.130:51375 (local address 127.13.75.190:40153)
0923 15:02:16.159671 (+ 100us) server_negotiation.cc:206] Beginning negotiation
0923 15:02:16.159673 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:02:16.159766 (+ 93us) server_negotiation.cc:407] Connection header received
0923 15:02:16.159870 (+ 104us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:02:16.159871 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:02:16.159884 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:02:16.159962 (+ 78us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:02:16.160301 (+ 339us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:16.160821 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:16.161551 (+ 730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:16.161753 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:16.161831 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:16.164606 (+ 2775us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.75.130:51375: BlockingRecv error: recv got EOF from 127.13.75.130:51375 (error 108)
Metrics: {"server-negotiator.queue_time_us":67}
Sep 23 15:02:16 dist-test-slave-cnb0 krb5kdc[21755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.75.131@KRBTEST.COM for oryx/127.13.75.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 15:02:16 dist-test-slave-cnb0 krb5kdc[21755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.75.131@KRBTEST.COM for oryx/127.13.75.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 15:02:16.303661 22773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:16.298467 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:40153 (local address 127.13.75.131:48463)
0923 15:02:16.298621 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0923 15:02:16.298632 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 15:02:16.298692 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:02:16.298962 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:02:16.298964 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:02:16.299130 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:02:16.299181 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:16.299183 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:16.299952 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:16.299953 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:16.300522 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:16.300525 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:16.300927 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:16.300928 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:16.300980 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:16.301485 (+ 505us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:02:16.301490 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:02:16.303601 (+ 2111us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.75.190:40153: Server oryx/127.13.75.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":91,"threads_started":1}
W20260923 15:02:16.303841 22771 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 15:02:16.298569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.131:48463 (local address 127.13.75.190:40153)
0923 15:02:16.298646 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0923 15:02:16.298648 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:02:16.298682 (+ 34us) server_negotiation.cc:407] Connection header received
0923 15:02:16.298752 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:02:16.298753 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:02:16.298766 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:02:16.298831 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:02:16.299300 (+ 469us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:16.299850 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:16.300643 (+ 793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:16.300838 (+ 195us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:16.300903 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:16.303775 (+ 2872us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.75.131:48463: BlockingRecv error: recv got EOF from 127.13.75.131:48463 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Sep 23 15:02:17 dist-test-slave-cnb0 krb5kdc[21755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.75.129@KRBTEST.COM for oryx/127.13.75.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 15:02:17 dist-test-slave-cnb0 krb5kdc[21755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.75.129@KRBTEST.COM for oryx/127.13.75.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 15:02:17.031992 22774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:17.026623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:40153 (local address 127.13.75.129:47725)
0923 15:02:17.026813 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0923 15:02:17.026826 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0923 15:02:17.026917 (+ 91us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:02:17.027113 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:02:17.027114 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:02:17.027296 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:02:17.027452 (+ 156us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:17.027456 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:17.028211 (+ 755us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:17.028212 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:17.028824 (+ 612us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:17.028827 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:17.029214 (+ 387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:17.029215 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:17.029264 (+ 49us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:17.029816 (+ 552us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:02:17.029828 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:02:17.031916 (+ 2088us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.75.190:40153: Server oryx/127.13.75.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":168,"thread_start_us":119,"threads_started":1}
W20260923 15:02:17.032157 22775 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 15:02:17.026736 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.129:47725 (local address 127.13.75.190:40153)
0923 15:02:17.026857 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0923 15:02:17.026859 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:02:17.026873 (+ 14us) server_negotiation.cc:407] Connection header received
0923 15:02:17.026991 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:02:17.026991 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:02:17.027003 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:02:17.027065 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:02:17.027552 (+ 487us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:17.028103 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:17.028920 (+ 817us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:17.029133 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:17.029196 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:17.032054 (+ 2858us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.75.129:47725: BlockingRecv error: recv got EOF from 127.13.75.129:47725 (error 108)
Metrics: {"server-negotiator.queue_time_us":103,"thread_start_us":53,"threads_started":1}
Sep 23 15:02:17 dist-test-slave-cnb0 krb5kdc[21755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.75.130@KRBTEST.COM for oryx/127.13.75.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 15:02:17 dist-test-slave-cnb0 krb5kdc[21755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.75.130@KRBTEST.COM for oryx/127.13.75.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 15:02:17.170996 22776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:17.165171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:40153 (local address 127.13.75.130:38905)
0923 15:02:17.165454 (+ 283us) negotiation.cc:107] Waiting for socket to connect
0923 15:02:17.165471 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0923 15:02:17.165537 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:02:17.165786 (+ 249us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:02:17.165787 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:02:17.165969 (+ 182us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:02:17.166020 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:17.166021 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:17.166770 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:17.166771 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:17.167377 (+ 606us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:17.167380 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:17.167793 (+ 413us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:17.167794 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:17.167846 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:17.168738 (+ 892us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:02:17.168743 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:02:17.170868 (+ 2125us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.75.190:40153: Server oryx/127.13.75.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":261,"thread_start_us":85,"threads_started":1}
W20260923 15:02:17.171176 22775 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 15:02:17.165373 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.130:38905 (local address 127.13.75.190:40153)
0923 15:02:17.165497 (+ 124us) server_negotiation.cc:206] Beginning negotiation
0923 15:02:17.165499 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:02:17.165520 (+ 21us) server_negotiation.cc:407] Connection header received
0923 15:02:17.165673 (+ 153us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:02:17.165675 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:02:17.165687 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:02:17.165735 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:02:17.166124 (+ 389us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:17.166676 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:17.167502 (+ 826us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:17.167709 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:17.167774 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:17.171085 (+ 3311us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.75.130:38905: BlockingRecv error: recv got EOF from 127.13.75.130:38905 (error 108)
Metrics: {"server-negotiator.queue_time_us":95}
Sep 23 15:02:17 dist-test-slave-cnb0 krb5kdc[21755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.75.131@KRBTEST.COM for oryx/127.13.75.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 15:02:17 dist-test-slave-cnb0 krb5kdc[21755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.75.131@KRBTEST.COM for oryx/127.13.75.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 15:02:17.310254 22777 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:17.304386 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:40153 (local address 127.13.75.131:50717)
0923 15:02:17.304644 (+ 258us) negotiation.cc:107] Waiting for socket to connect
0923 15:02:17.304659 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0923 15:02:17.304747 (+ 88us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:02:17.304935 (+ 188us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:02:17.304937 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:02:17.305144 (+ 207us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:02:17.305197 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:17.305199 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:17.306079 (+ 880us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:17.306080 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:17.306976 (+ 896us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:17.306980 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:17.307394 (+ 414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:17.307395 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:17.307449 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:17.307962 (+ 513us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:02:17.307966 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:02:17.310189 (+ 2223us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.75.190:40153: Server oryx/127.13.75.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":235,"thread_start_us":119,"threads_started":1}
W20260923 15:02:17.310388 22775 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 15:02:17.304515 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.131:50717 (local address 127.13.75.190:40153)
0923 15:02:17.304629 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0923 15:02:17.304632 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:02:17.304724 (+ 92us) server_negotiation.cc:407] Connection header received
0923 15:02:17.304816 (+ 92us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:02:17.304817 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:02:17.304834 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:02:17.304928 (+ 94us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:02:17.305298 (+ 370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:17.305923 (+ 625us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:17.307098 (+ 1175us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:17.307295 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:17.307363 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:17.310332 (+ 2969us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.75.131:50717: BlockingRecv error: recv got EOF from 127.13.75.131:50717 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
Sep 23 15:02:18 dist-test-slave-cnb0 krb5kdc[21755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.75.129@KRBTEST.COM for oryx/127.13.75.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 15:02:18 dist-test-slave-cnb0 krb5kdc[21755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.75.129@KRBTEST.COM for oryx/127.13.75.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 15:02:18.039265 22778 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:18.032660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:40153 (local address 127.13.75.129:54349)
0923 15:02:18.033094 (+ 434us) negotiation.cc:107] Waiting for socket to connect
0923 15:02:18.033109 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0923 15:02:18.033190 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:02:18.033418 (+ 228us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:02:18.033420 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:02:18.033759 (+ 339us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:02:18.033812 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:18.033815 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:18.034616 (+ 801us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:18.034618 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:18.035581 (+ 963us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:18.035584 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:18.036189 (+ 605us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:18.036190 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:18.036238 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:18.036940 (+ 702us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:02:18.036945 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:02:18.039201 (+ 2256us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.75.190:40153: Server oryx/127.13.75.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":415,"thread_start_us":234,"threads_started":1}
W20260923 15:02:18.039552 22779 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 15:02:18.032806 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.129:54349 (local address 127.13.75.190:40153)
0923 15:02:18.032999 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0923 15:02:18.033002 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:02:18.033215 (+ 213us) server_negotiation.cc:407] Connection header received
0923 15:02:18.033250 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:02:18.033251 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:02:18.033264 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:02:18.033371 (+ 107us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:02:18.033931 (+ 560us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:18.034464 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:18.035695 (+ 1231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:18.036025 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:18.036103 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:18.039473 (+ 3370us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.75.129:54349: BlockingRecv error: recv got EOF from 127.13.75.129:54349 (error 108)
Metrics: {"server-negotiator.queue_time_us":163,"thread_start_us":111,"threads_started":1}
Sep 23 15:02:18 dist-test-slave-cnb0 krb5kdc[21755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.75.130@KRBTEST.COM for oryx/127.13.75.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 15:02:18 dist-test-slave-cnb0 krb5kdc[21755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.75.130@KRBTEST.COM for oryx/127.13.75.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 15:02:18.176893 22780 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:18.171689 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:40153 (local address 127.13.75.130:55673)
0923 15:02:18.171872 (+ 183us) negotiation.cc:107] Waiting for socket to connect
0923 15:02:18.171886 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0923 15:02:18.171980 (+ 94us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:02:18.172125 (+ 145us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:02:18.172126 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:02:18.172313 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:02:18.172361 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:18.172363 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:18.173107 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:18.173108 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:18.173640 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:18.173641 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:18.174092 (+ 451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:18.174093 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:18.174161 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:18.174690 (+ 529us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:02:18.174695 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:02:18.176830 (+ 2135us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.75.190:40153: Server oryx/127.13.75.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":90,"threads_started":1}
W20260923 15:02:18.177131 22779 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 15:02:18.171782 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.130:55673 (local address 127.13.75.190:40153)
0923 15:02:18.171859 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0923 15:02:18.171862 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:02:18.171960 (+ 98us) server_negotiation.cc:407] Connection header received
0923 15:02:18.172043 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:02:18.172044 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:02:18.172058 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:02:18.172109 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:02:18.172447 (+ 338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:18.173006 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:18.173770 (+ 764us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:18.173975 (+ 205us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:18.174078 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:18.177082 (+ 3004us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.75.130:55673: BlockingRecv error: recv got EOF from 127.13.75.130:55673 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Sep 23 15:02:18 dist-test-slave-cnb0 krb5kdc[21755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.75.131@KRBTEST.COM for oryx/127.13.75.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 15:02:18 dist-test-slave-cnb0 krb5kdc[21755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.75.131@KRBTEST.COM for oryx/127.13.75.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 15:02:18.316174 22781 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:18.311095 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:40153 (local address 127.13.75.131:42137)
0923 15:02:18.311270 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0923 15:02:18.311281 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 15:02:18.311346 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:02:18.311562 (+ 216us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:02:18.311563 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:02:18.311728 (+ 165us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:02:18.311776 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:18.311778 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:18.312494 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:18.312495 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:18.313076 (+ 581us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:18.313078 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:18.313428 (+ 350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:18.313429 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:18.313479 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:18.313927 (+ 448us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:02:18.313932 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:02:18.316103 (+ 2171us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.75.190:40153: Server oryx/127.13.75.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":79,"threads_started":1}
W20260923 15:02:18.316346 22779 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 15:02:18.311161 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.131:42137 (local address 127.13.75.190:40153)
0923 15:02:18.311250 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0923 15:02:18.311252 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:02:18.311396 (+ 144us) server_negotiation.cc:407] Connection header received
0923 15:02:18.311434 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:02:18.311435 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:02:18.311450 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:02:18.311514 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:02:18.311882 (+ 368us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:18.312405 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:18.313158 (+ 753us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:18.313348 (+ 190us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:18.313406 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:18.316271 (+ 2865us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.75.131:42137: BlockingRecv error: recv got EOF from 127.13.75.131:42137 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
Sep 23 15:02:19 dist-test-slave-cnb0 krb5kdc[21755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.75.129@KRBTEST.COM for oryx/127.13.75.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 15:02:19 dist-test-slave-cnb0 krb5kdc[21755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.75.129@KRBTEST.COM for oryx/127.13.75.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 15:02:19.045524 22782 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:19.039856 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:40153 (local address 127.13.75.129:37269)
0923 15:02:19.040004 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0923 15:02:19.040022 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0923 15:02:19.040090 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:02:19.040251 (+ 161us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:02:19.040252 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:02:19.040433 (+ 181us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:02:19.040493 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:19.040495 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:19.041356 (+ 861us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:19.041357 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:19.042013 (+ 656us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:19.042015 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:19.042478 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:19.042479 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:19.042587 (+ 108us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:19.043128 (+ 541us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:02:19.043133 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:02:19.045449 (+ 2316us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.75.190:40153: Server oryx/127.13.75.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":89,"threads_started":1}
W20260923 15:02:19.045825 22783 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 15:02:19.039956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.129:37269 (local address 127.13.75.190:40153)
0923 15:02:19.040107 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0923 15:02:19.040109 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:02:19.040122 (+ 13us) server_negotiation.cc:407] Connection header received
0923 15:02:19.040149 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:02:19.040149 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:02:19.040160 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:02:19.040235 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:02:19.040666 (+ 431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:19.041196 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:19.042144 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:19.042376 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:19.042451 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:19.045609 (+ 3158us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.75.129:37269: BlockingRecv error: recv got EOF from 127.13.75.129:37269 (error 108)
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":92,"threads_started":1}
Sep 23 15:02:19 dist-test-slave-cnb0 krb5kdc[21755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.75.130@KRBTEST.COM for oryx/127.13.75.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 15:02:19 dist-test-slave-cnb0 krb5kdc[21755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.75.130@KRBTEST.COM for oryx/127.13.75.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 15:02:19.182502 22784 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:19.177607 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:40153 (local address 127.13.75.130:40317)
0923 15:02:19.177780 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0923 15:02:19.177798 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0923 15:02:19.177858 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:02:19.178011 (+ 153us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:02:19.178012 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:02:19.178175 (+ 163us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:02:19.178226 (+ 51us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:19.178227 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:19.179002 (+ 775us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:19.179003 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:19.179531 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:19.179533 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:19.179927 (+ 394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:19.179928 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:19.179983 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:19.180539 (+ 556us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:02:19.180543 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:02:19.182434 (+ 1891us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.75.190:40153: Server oryx/127.13.75.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":110,"threads_started":1}
W20260923 15:02:19.182713 22783 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 15:02:19.177710 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.130:40317 (local address 127.13.75.190:40153)
0923 15:02:19.177817 (+ 107us) server_negotiation.cc:206] Beginning negotiation
0923 15:02:19.177819 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:02:19.177845 (+ 26us) server_negotiation.cc:407] Connection header received
0923 15:02:19.177926 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:02:19.177927 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:02:19.177939 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:02:19.177998 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:02:19.178297 (+ 299us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:19.178811 (+ 514us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:19.179633 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:19.179827 (+ 194us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:19.179887 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:19.182646 (+ 2759us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.75.130:40317: BlockingRecv error: recv got EOF from 127.13.75.130:40317 (error 108)
Metrics: {"server-negotiator.queue_time_us":73}
Sep 23 15:02:19 dist-test-slave-cnb0 krb5kdc[21755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.75.131@KRBTEST.COM for oryx/127.13.75.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 15:02:19 dist-test-slave-cnb0 krb5kdc[21755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.75.131@KRBTEST.COM for oryx/127.13.75.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 15:02:19.322118 22785 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:19.316783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:40153 (local address 127.13.75.131:51707)
0923 15:02:19.316931 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0923 15:02:19.316948 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0923 15:02:19.317054 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:02:19.317249 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:02:19.317250 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:02:19.317517 (+ 267us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:02:19.317588 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:19.317590 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:19.318403 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:19.318404 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:19.319011 (+ 607us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:19.319014 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:19.319509 (+ 495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:19.319510 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:19.319558 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:19.320135 (+ 577us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:02:19.320141 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:02:19.322055 (+ 1914us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.75.190:40153: Server oryx/127.13.75.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":72,"threads_started":1}
W20260923 15:02:19.322356 22783 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 15:02:19.316850 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.131:51707 (local address 127.13.75.190:40153)
0923 15:02:19.316906 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0923 15:02:19.316909 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:02:19.317028 (+ 119us) server_negotiation.cc:407] Connection header received
0923 15:02:19.317143 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:02:19.317143 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:02:19.317161 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:02:19.317227 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:02:19.317710 (+ 483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:19.318272 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:19.319116 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:19.319417 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:19.319487 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:19.322226 (+ 2739us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.75.131:51707: BlockingRecv error: recv got EOF from 127.13.75.131:51707 (error 108)
Metrics: {"server-negotiator.queue_time_us":31}
Sep 23 15:02:20 dist-test-slave-cnb0 krb5kdc[21755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.75.129@KRBTEST.COM for oryx/127.13.75.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 15:02:20 dist-test-slave-cnb0 krb5kdc[21755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.75.129@KRBTEST.COM for oryx/127.13.75.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 15:02:20.052320 22786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:20.046334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:40153 (local address 127.13.75.129:59913)
0923 15:02:20.046554 (+ 220us) negotiation.cc:107] Waiting for socket to connect
0923 15:02:20.046594 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0923 15:02:20.046672 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:02:20.046988 (+ 316us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:02:20.046990 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:02:20.047175 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:02:20.047225 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:20.047227 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:20.048180 (+ 953us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:20.048181 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:20.048722 (+ 541us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:20.048724 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:20.049110 (+ 386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:20.049111 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:20.049179 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:20.049691 (+ 512us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:02:20.049697 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:02:20.052233 (+ 2536us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.75.190:40153: Server oryx/127.13.75.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":101,"threads_started":1}
W20260923 15:02:20.052482 22787 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 15:02:20.046611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.129:59913 (local address 127.13.75.190:40153)
0923 15:02:20.046787 (+ 176us) server_negotiation.cc:206] Beginning negotiation
0923 15:02:20.046790 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:02:20.046798 (+ 8us) server_negotiation.cc:407] Connection header received
0923 15:02:20.046837 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:02:20.046838 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:02:20.046851 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:02:20.046973 (+ 122us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:02:20.047531 (+ 558us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:20.048083 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:20.048816 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:20.049032 (+ 216us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:20.049111 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:20.052387 (+ 3276us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.75.129:59913: BlockingRecv error: recv got EOF from 127.13.75.129:59913 (error 108)
Metrics: {"server-negotiator.queue_time_us":154,"thread_start_us":64,"threads_started":1}
Sep 23 15:02:20 dist-test-slave-cnb0 krb5kdc[21755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.75.130@KRBTEST.COM for oryx/127.13.75.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 15:02:20 dist-test-slave-cnb0 krb5kdc[21755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.75.130@KRBTEST.COM for oryx/127.13.75.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 15:02:20.188737 22788 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:20.183242 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:40153 (local address 127.13.75.130:51313)
0923 15:02:20.183485 (+ 243us) negotiation.cc:107] Waiting for socket to connect
0923 15:02:20.183500 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0923 15:02:20.183642 (+ 142us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:02:20.183828 (+ 186us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:02:20.183830 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:02:20.184017 (+ 187us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:02:20.184079 (+ 62us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:20.184080 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:20.184964 (+ 884us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:20.184966 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:20.185537 (+ 571us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:20.185540 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:20.185890 (+ 350us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:20.185892 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:20.185959 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:20.186525 (+ 566us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:02:20.186530 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:02:20.188637 (+ 2107us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.75.190:40153: Server oryx/127.13.75.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":196,"threads_started":1}
W20260923 15:02:20.188791 22787 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 15:02:20.183350 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.130:51313 (local address 127.13.75.190:40153)
0923 15:02:20.183441 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0923 15:02:20.183444 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:02:20.183660 (+ 216us) server_negotiation.cc:407] Connection header received
0923 15:02:20.183700 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:02:20.183701 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:02:20.183716 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:02:20.183788 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:02:20.184179 (+ 391us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:20.184854 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:20.185615 (+ 761us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:20.185799 (+ 184us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:20.185852 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:20.188724 (+ 2872us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.75.130:51313: BlockingRecv error: recv got EOF from 127.13.75.130:51313 (error 108)
Metrics: {"server-negotiator.queue_time_us":52}
Sep 23 15:02:20 dist-test-slave-cnb0 krb5kdc[21755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.75.131@KRBTEST.COM for oryx/127.13.75.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 15:02:20 dist-test-slave-cnb0 krb5kdc[21755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.75.131@KRBTEST.COM for oryx/127.13.75.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 15:02:20.328020 22789 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:20.322734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:40153 (local address 127.13.75.131:52719)
0923 15:02:20.323011 (+ 277us) negotiation.cc:107] Waiting for socket to connect
0923 15:02:20.323024 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0923 15:02:20.323095 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:02:20.323253 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:02:20.323254 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:02:20.323439 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:02:20.323494 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:20.323496 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:20.324170 (+ 674us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:20.324171 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:20.324704 (+ 533us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:20.324706 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:20.325078 (+ 372us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:20.325079 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:20.325139 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:20.325623 (+ 484us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:02:20.325628 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:02:20.327907 (+ 2279us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.75.190:40153: Server oryx/127.13.75.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":258,"thread_start_us":120,"threads_started":1}
W20260923 15:02:20.328084 22787 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 15:02:20.322780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.131:52719 (local address 127.13.75.190:40153)
0923 15:02:20.322873 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0923 15:02:20.322875 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:02:20.323076 (+ 201us) server_negotiation.cc:407] Connection header received
0923 15:02:20.323150 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:02:20.323151 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:02:20.323167 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:02:20.323247 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:02:20.323583 (+ 336us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:20.324082 (+ 499us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:20.324803 (+ 721us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:20.325000 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:20.325080 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:20.328010 (+ 2930us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.75.131:52719: BlockingRecv error: recv got EOF from 127.13.75.131:52719 (error 108)
Metrics: {"server-negotiator.queue_time_us":46}
Sep 23 15:02:21 dist-test-slave-cnb0 krb5kdc[21755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.75.129@KRBTEST.COM for oryx/127.13.75.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 15:02:21 dist-test-slave-cnb0 krb5kdc[21755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.75.129@KRBTEST.COM for oryx/127.13.75.190@KRBTEST.COM, Server not found in Kerberos database
W20260923 15:02:21.062875 22791 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 15:02:21.052940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.129:57745 (local address 127.13.75.190:40153)
0923 15:02:21.053087 (+ 147us) server_negotiation.cc:206] Beginning negotiation
0923 15:02:21.053090 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:02:21.053099 (+ 9us) server_negotiation.cc:407] Connection header received
0923 15:02:21.053125 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:02:21.053125 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:02:21.053137 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:02:21.053200 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:02:21.053515 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:21.054081 (+ 566us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:21.054793 (+ 712us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:21.055029 (+ 236us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:21.055087 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:21.062796 (+ 7709us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.75.129:57745: BlockingRecv error: recv got EOF from 127.13.75.129:57745 (error 108)
Metrics: {"server-negotiator.queue_time_us":126,"thread_start_us":72,"threads_started":1}
I20260923 15:02:21.062832 22790 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:21.052818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:40153 (local address 127.13.75.129:57745)
0923 15:02:21.052970 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0923 15:02:21.052984 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0923 15:02:21.053077 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:02:21.053220 (+ 143us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:02:21.053221 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:02:21.053385 (+ 164us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:02:21.053437 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:21.053439 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:21.054160 (+ 721us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:21.054161 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:21.054708 (+ 547us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:21.054710 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:21.055097 (+ 387us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:21.055098 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:21.055144 (+ 46us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:21.060169 (+ 5025us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:02:21.060176 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:02:21.062696 (+ 2520us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.75.190:40153: Server oryx/127.13.75.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":65,"threads_started":1}
Sep 23 15:02:21 dist-test-slave-cnb0 krb5kdc[21755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.75.130@KRBTEST.COM for oryx/127.13.75.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 15:02:21 dist-test-slave-cnb0 krb5kdc[21755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.75.130@KRBTEST.COM for oryx/127.13.75.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 15:02:21.194646 22792 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:21.189256 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:40153 (local address 127.13.75.130:50821)
0923 15:02:21.189412 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0923 15:02:21.189424 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 15:02:21.189536 (+ 112us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:02:21.189714 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:02:21.189715 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:02:21.189924 (+ 209us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:02:21.189972 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:21.189974 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:21.190735 (+ 761us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:21.190737 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:21.191358 (+ 621us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:21.191360 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:21.191774 (+ 414us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:21.191775 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:21.191837 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:21.192385 (+ 548us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:02:21.192391 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:02:21.194504 (+ 2113us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.75.190:40153: Server oryx/127.13.75.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":83,"threads_started":1}
W20260923 15:02:21.194676 22791 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 15:02:21.189427 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.130:50821 (local address 127.13.75.190:40153)
0923 15:02:21.189520 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0923 15:02:21.189523 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:02:21.189535 (+ 12us) server_negotiation.cc:407] Connection header received
0923 15:02:21.189607 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:02:21.189608 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:02:21.189623 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:02:21.189708 (+ 85us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:02:21.190073 (+ 365us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:21.190636 (+ 563us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:21.191457 (+ 821us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:21.191678 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:21.191775 (+ 97us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:21.194624 (+ 2849us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.75.130:50821: BlockingRecv error: recv got EOF from 127.13.75.130:50821 (error 108)
Metrics: {"server-negotiator.queue_time_us":63}
Sep 23 15:02:21 dist-test-slave-cnb0 krb5kdc[21755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.75.131@KRBTEST.COM for oryx/127.13.75.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 15:02:21 dist-test-slave-cnb0 krb5kdc[21755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.75.131@KRBTEST.COM for oryx/127.13.75.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 15:02:21.333811 22793 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:21.328718 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:40153 (local address 127.13.75.131:47303)
0923 15:02:21.328874 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0923 15:02:21.328888 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0923 15:02:21.328957 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:02:21.329156 (+ 199us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:02:21.329157 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:02:21.329342 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:02:21.329388 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:21.329389 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:21.330120 (+ 731us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:21.330122 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:21.330653 (+ 531us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:21.330654 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:21.331101 (+ 447us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:21.331102 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:21.331167 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:21.331791 (+ 624us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:02:21.331797 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:02:21.333749 (+ 1952us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.75.190:40153: Server oryx/127.13.75.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":94,"threads_started":1}
W20260923 15:02:21.333961 22791 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 15:02:21.328853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.131:47303 (local address 127.13.75.190:40153)
0923 15:02:21.328935 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0923 15:02:21.328937 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:02:21.328949 (+ 12us) server_negotiation.cc:407] Connection header received
0923 15:02:21.329066 (+ 117us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:02:21.329067 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:02:21.329085 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:02:21.329146 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:02:21.329516 (+ 370us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:21.330023 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:21.330740 (+ 717us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:21.331005 (+ 265us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:21.331090 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:21.333866 (+ 2776us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.75.131:47303: BlockingRecv error: recv got EOF from 127.13.75.131:47303 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Sep 23 15:02:22 dist-test-slave-cnb0 krb5kdc[21755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.75.129@KRBTEST.COM for oryx/127.13.75.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 15:02:22 dist-test-slave-cnb0 krb5kdc[21755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.75.129@KRBTEST.COM for oryx/127.13.75.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 15:02:22.069574 22794 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:22.063534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:40153 (local address 127.13.75.129:43383)
0923 15:02:22.063705 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0923 15:02:22.063718 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0923 15:02:22.063782 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:02:22.064053 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:02:22.064055 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:02:22.064222 (+ 167us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:02:22.064275 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:22.064276 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:22.065047 (+ 771us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:22.065049 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:22.065853 (+ 804us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:22.065866 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:22.066398 (+ 532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:22.066400 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:22.066462 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:22.067070 (+ 608us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:02:22.067076 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:02:22.069506 (+ 2430us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.75.190:40153: Server oryx/127.13.75.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":87,"threads_started":1}
W20260923 15:02:22.069739 22795 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 15:02:22.063674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.129:43383 (local address 127.13.75.190:40153)
0923 15:02:22.063857 (+ 183us) server_negotiation.cc:206] Beginning negotiation
0923 15:02:22.063859 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:02:22.063870 (+ 11us) server_negotiation.cc:407] Connection header received
0923 15:02:22.063897 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:02:22.063898 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:02:22.063909 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:02:22.064009 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:02:22.064399 (+ 390us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:22.064926 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:22.065992 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:22.066287 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:22.066388 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:22.069638 (+ 3250us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.75.129:43383: BlockingRecv error: recv got EOF from 127.13.75.129:43383 (error 108)
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":65,"threads_started":1}
Sep 23 15:02:22 dist-test-slave-cnb0 krb5kdc[21755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.75.130@KRBTEST.COM for oryx/127.13.75.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 15:02:22 dist-test-slave-cnb0 krb5kdc[21755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.75.130@KRBTEST.COM for oryx/127.13.75.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 15:02:22.200766 22796 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:22.195215 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:40153 (local address 127.13.75.130:35637)
0923 15:02:22.195386 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0923 15:02:22.195407 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0923 15:02:22.195470 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:02:22.195648 (+ 178us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:02:22.195649 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:02:22.195827 (+ 178us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:02:22.195881 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:22.195882 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:22.196626 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:22.196628 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:22.197252 (+ 624us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:22.197255 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:22.197775 (+ 520us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:22.197776 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:22.197832 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:22.198408 (+ 576us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:02:22.198414 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:02:22.200701 (+ 2287us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.75.190:40153: Server oryx/127.13.75.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":111,"threads_started":1}
W20260923 15:02:22.200945 22795 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 15:02:22.195344 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.130:35637 (local address 127.13.75.190:40153)
0923 15:02:22.195409 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0923 15:02:22.195411 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:02:22.195462 (+ 51us) server_negotiation.cc:407] Connection header received
0923 15:02:22.195529 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:02:22.195530 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:02:22.195541 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:02:22.195608 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:02:22.195953 (+ 345us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:22.196512 (+ 559us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:22.197429 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:22.197647 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:22.197731 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:22.200863 (+ 3132us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.75.130:35637: BlockingRecv error: recv got EOF from 127.13.75.130:35637 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
Sep 23 15:02:22 dist-test-slave-cnb0 krb5kdc[21755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.75.131@KRBTEST.COM for oryx/127.13.75.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 15:02:22 dist-test-slave-cnb0 krb5kdc[21755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.75.131@KRBTEST.COM for oryx/127.13.75.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 15:02:22.339980 22797 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:22.334433 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:40153 (local address 127.13.75.131:36079)
0923 15:02:22.334602 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0923 15:02:22.334620 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0923 15:02:22.334685 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:02:22.334976 (+ 291us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:02:22.334977 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:02:22.335156 (+ 179us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:02:22.335219 (+ 63us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:22.335222 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:22.336057 (+ 835us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:22.336058 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:22.336594 (+ 536us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:22.336596 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:22.337040 (+ 444us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:22.337042 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:22.337131 (+ 89us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:22.337745 (+ 614us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:02:22.337751 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:02:22.339849 (+ 2098us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.75.190:40153: Server oryx/127.13.75.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":100,"threads_started":1}
W20260923 15:02:22.340001 22795 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 15:02:22.334520 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.131:36079 (local address 127.13.75.190:40153)
0923 15:02:22.334608 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0923 15:02:22.334611 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:02:22.334688 (+ 77us) server_negotiation.cc:407] Connection header received
0923 15:02:22.334740 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:02:22.334740 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:02:22.334753 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:02:22.334859 (+ 106us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:02:22.335393 (+ 534us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:22.335906 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:22.336687 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:22.336920 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:22.337034 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:22.339941 (+ 2907us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.75.131:36079: BlockingRecv error: recv got EOF from 127.13.75.131:36079 (error 108)
Metrics: {"server-negotiator.queue_time_us":58}
Sep 23 15:02:23 dist-test-slave-cnb0 krb5kdc[21755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.75.129@KRBTEST.COM for oryx/127.13.75.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 15:02:23 dist-test-slave-cnb0 krb5kdc[21755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.75.129@KRBTEST.COM for oryx/127.13.75.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 15:02:23.075981 22798 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:23.070322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:40153 (local address 127.13.75.129:37167)
0923 15:02:23.070473 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0923 15:02:23.070494 (+ 21us) client_negotiation.cc:174] Beginning negotiation
0923 15:02:23.070550 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:02:23.070776 (+ 226us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:02:23.070777 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:02:23.071041 (+ 264us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:02:23.071107 (+ 66us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:23.071109 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:23.071888 (+ 779us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:23.071889 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:23.072444 (+ 555us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:23.072446 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:23.072865 (+ 419us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:23.072867 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:23.072937 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:23.073440 (+ 503us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:02:23.073445 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:02:23.075897 (+ 2452us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.75.190:40153: Server oryx/127.13.75.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":99,"threads_started":1}
W20260923 15:02:23.076205 22799 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 15:02:23.070472 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.129:37167 (local address 127.13.75.190:40153)
0923 15:02:23.070617 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0923 15:02:23.070619 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:02:23.070629 (+ 10us) server_negotiation.cc:407] Connection header received
0923 15:02:23.070657 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:02:23.070657 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:02:23.070669 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:02:23.070745 (+ 76us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:02:23.071240 (+ 495us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:23.071778 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:23.072532 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:23.072753 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:23.072859 (+ 106us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:23.076095 (+ 3236us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.75.129:37167: BlockingRecv error: recv got EOF from 127.13.75.129:37167 (error 108)
Metrics: {"server-negotiator.queue_time_us":117,"thread_start_us":86,"threads_started":1}
Sep 23 15:02:23 dist-test-slave-cnb0 krb5kdc[21755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.75.130@KRBTEST.COM for oryx/127.13.75.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 15:02:23 dist-test-slave-cnb0 krb5kdc[21755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.75.130@KRBTEST.COM for oryx/127.13.75.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 15:02:23.206470 22800 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:23.201355 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:40153 (local address 127.13.75.130:56711)
0923 15:02:23.201500 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0923 15:02:23.201513 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0923 15:02:23.201576 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:02:23.201730 (+ 154us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:02:23.201731 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:02:23.201927 (+ 196us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:02:23.201975 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:23.201977 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:23.202658 (+ 681us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:23.202659 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:23.203267 (+ 608us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:23.203269 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:23.203709 (+ 440us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:23.203710 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:23.203830 (+ 120us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:23.204309 (+ 479us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:02:23.204314 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:02:23.206396 (+ 2082us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.75.190:40153: Server oryx/127.13.75.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":90,"threads_started":1}
W20260923 15:02:23.206610 22799 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 15:02:23.201514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.130:56711 (local address 127.13.75.190:40153)
0923 15:02:23.201582 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0923 15:02:23.201585 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:02:23.201597 (+ 12us) server_negotiation.cc:407] Connection header received
0923 15:02:23.201647 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:02:23.201648 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:02:23.201659 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:02:23.201721 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:02:23.202063 (+ 342us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:23.202568 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:23.203382 (+ 814us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:23.203584 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:23.203707 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:23.206512 (+ 2805us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.75.130:56711: BlockingRecv error: recv got EOF from 127.13.75.130:56711 (error 108)
Metrics: {"server-negotiator.queue_time_us":37}
Sep 23 15:02:23 dist-test-slave-cnb0 krb5kdc[21755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.75.131@KRBTEST.COM for oryx/127.13.75.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 15:02:23 dist-test-slave-cnb0 krb5kdc[21755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.75.131@KRBTEST.COM for oryx/127.13.75.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 15:02:23.345772 22801 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:23.340576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:40153 (local address 127.13.75.131:50279)
0923 15:02:23.340814 (+ 238us) negotiation.cc:107] Waiting for socket to connect
0923 15:02:23.340828 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0923 15:02:23.340925 (+ 97us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:02:23.341097 (+ 172us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:02:23.341098 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:02:23.341290 (+ 192us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:02:23.341334 (+ 44us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:23.341336 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:23.342045 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:23.342046 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:23.342615 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:23.342617 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:23.343051 (+ 434us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:23.343052 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:23.343116 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:23.343629 (+ 513us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:02:23.343633 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:02:23.345672 (+ 2039us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.75.190:40153: Server oryx/127.13.75.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":220,"thread_start_us":138,"threads_started":1}
W20260923 15:02:23.345825 22799 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 15:02:23.340702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.131:50279 (local address 127.13.75.190:40153)
0923 15:02:23.340787 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0923 15:02:23.340790 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:02:23.340907 (+ 117us) server_negotiation.cc:407] Connection header received
0923 15:02:23.340988 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:02:23.340990 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:02:23.341008 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:02:23.341068 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:02:23.341422 (+ 354us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:23.341948 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:23.342708 (+ 760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:23.342962 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:23.343037 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:23.345759 (+ 2722us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.75.131:50279: BlockingRecv error: recv got EOF from 127.13.75.131:50279 (error 108)
Metrics: {"server-negotiator.queue_time_us":36}
Sep 23 15:02:24 dist-test-slave-cnb0 krb5kdc[21755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.75.129@KRBTEST.COM for oryx/127.13.75.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 15:02:24 dist-test-slave-cnb0 krb5kdc[21755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.75.129@KRBTEST.COM for oryx/127.13.75.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 15:02:24.081583 22802 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:24.076563 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:40153 (local address 127.13.75.129:57125)
0923 15:02:24.076749 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0923 15:02:24.076764 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0923 15:02:24.076838 (+ 74us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:02:24.077074 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:02:24.077075 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:02:24.077246 (+ 171us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:02:24.077294 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:24.077295 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:24.078033 (+ 738us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:24.078034 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:24.078562 (+ 528us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:24.078564 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:24.079033 (+ 469us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:24.079034 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:24.079078 (+ 44us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:24.079633 (+ 555us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:02:24.079638 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:02:24.081493 (+ 1855us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.75.190:40153: Server oryx/127.13.75.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":84,"threads_started":1}
W20260923 15:02:24.081755 22803 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 15:02:24.076693 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.129:57125 (local address 127.13.75.190:40153)
0923 15:02:24.076884 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0923 15:02:24.076886 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:02:24.076895 (+ 9us) server_negotiation.cc:407] Connection header received
0923 15:02:24.076918 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:02:24.076918 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:02:24.076929 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:02:24.077002 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:02:24.077425 (+ 423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:24.077948 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:24.078649 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:24.078864 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:24.079013 (+ 149us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:24.081642 (+ 2629us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.75.129:57125: BlockingRecv error: recv got EOF from 127.13.75.129:57125 (error 108)
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":79,"threads_started":1}
Sep 23 15:02:24 dist-test-slave-cnb0 krb5kdc[21755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.75.130@KRBTEST.COM for oryx/127.13.75.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 15:02:24 dist-test-slave-cnb0 krb5kdc[21755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.75.130@KRBTEST.COM for oryx/127.13.75.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 15:02:24.212504 22804 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:24.207108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:40153 (local address 127.13.75.130:60517)
0923 15:02:24.207303 (+ 195us) negotiation.cc:107] Waiting for socket to connect
0923 15:02:24.207316 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0923 15:02:24.207443 (+ 127us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:02:24.207603 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:02:24.207604 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:02:24.207781 (+ 177us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:02:24.207831 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:24.207833 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:24.208577 (+ 744us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:24.208578 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:24.209110 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:24.209112 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:24.209596 (+ 484us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:24.209597 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:24.209687 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:24.210309 (+ 622us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:02:24.210314 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:02:24.212443 (+ 2129us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.75.190:40153: Server oryx/127.13.75.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":88,"threads_started":1}
W20260923 15:02:24.212687 22803 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 15:02:24.207235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.130:60517 (local address 127.13.75.190:40153)
0923 15:02:24.207289 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0923 15:02:24.207291 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:02:24.207459 (+ 168us) server_negotiation.cc:407] Connection header received
0923 15:02:24.207500 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:02:24.207501 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:02:24.207516 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:02:24.207583 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:02:24.207915 (+ 332us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:24.208467 (+ 552us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:24.209246 (+ 779us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:24.209468 (+ 222us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:24.209596 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:24.212607 (+ 3011us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.75.130:60517: BlockingRecv error: recv got EOF from 127.13.75.130:60517 (error 108)
Metrics: {"server-negotiator.queue_time_us":26}
Sep 23 15:02:24 dist-test-slave-cnb0 krb5kdc[21755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.75.131@KRBTEST.COM for oryx/127.13.75.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 15:02:24 dist-test-slave-cnb0 krb5kdc[21755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.75.131@KRBTEST.COM for oryx/127.13.75.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 15:02:24.352095 22805 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:24.346314 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:40153 (local address 127.13.75.131:41025)
0923 15:02:24.346491 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0923 15:02:24.346506 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0923 15:02:24.346608 (+ 102us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:02:24.346783 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:02:24.346784 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:02:24.347046 (+ 262us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:02:24.347094 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:24.347096 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:24.347865 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:24.347866 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:24.348461 (+ 595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:24.348463 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:24.348866 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:24.348867 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:24.348943 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:24.349493 (+ 550us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:02:24.349498 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:02:24.351988 (+ 2490us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.75.190:40153: Server oryx/127.13.75.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":65,"threads_started":1}
W20260923 15:02:24.352193 22803 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 15:02:24.346539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.131:41025 (local address 127.13.75.190:40153)
0923 15:02:24.346624 (+ 85us) server_negotiation.cc:206] Beginning negotiation
0923 15:02:24.346627 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:02:24.346636 (+ 9us) server_negotiation.cc:407] Connection header received
0923 15:02:24.346667 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:02:24.346668 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:02:24.346685 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:02:24.346773 (+ 88us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:02:24.347196 (+ 423us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:24.347763 (+ 567us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:24.348560 (+ 797us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:24.348777 (+ 217us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:24.348866 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:24.352098 (+ 3232us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.75.131:41025: BlockingRecv error: recv got EOF from 127.13.75.131:41025 (error 108)
Metrics: {"server-negotiator.queue_time_us":53}
Sep 23 15:02:25 dist-test-slave-cnb0 krb5kdc[21755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.75.129@KRBTEST.COM for oryx/127.13.75.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 15:02:25 dist-test-slave-cnb0 krb5kdc[21755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.75.129@KRBTEST.COM for oryx/127.13.75.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 15:02:25.087919 22806 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:25.082260 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:40153 (local address 127.13.75.129:59643)
0923 15:02:25.082426 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0923 15:02:25.082437 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 15:02:25.082565 (+ 128us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:02:25.082763 (+ 198us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:02:25.082765 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:02:25.082996 (+ 231us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:02:25.083049 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:25.083050 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:25.083831 (+ 781us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:25.083833 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:25.084487 (+ 654us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:25.084490 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:25.084921 (+ 431us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:25.084923 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:25.084985 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:25.085522 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:02:25.085527 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:02:25.087795 (+ 2268us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.75.190:40153: Server oryx/127.13.75.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":107,"threads_started":1}
W20260923 15:02:25.088001 22807 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 15:02:25.082301 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.129:59643 (local address 127.13.75.190:40153)
0923 15:02:25.082511 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0923 15:02:25.082513 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:02:25.082530 (+ 17us) server_negotiation.cc:407] Connection header received
0923 15:02:25.082646 (+ 116us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:02:25.082647 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:02:25.082662 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:02:25.082746 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:02:25.083168 (+ 422us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:25.083722 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:25.084606 (+ 884us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:25.084818 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:25.084889 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:25.087907 (+ 3018us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.75.129:59643: BlockingRecv error: recv got EOF from 127.13.75.129:59643 (error 108)
Metrics: {"server-negotiator.queue_time_us":185,"thread_start_us":142,"threads_started":1}
Sep 23 15:02:25 dist-test-slave-cnb0 krb5kdc[21755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.75.130@KRBTEST.COM for oryx/127.13.75.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 15:02:25 dist-test-slave-cnb0 krb5kdc[21755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.75.130@KRBTEST.COM for oryx/127.13.75.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 15:02:25.218307 22808 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:25.213169 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:40153 (local address 127.13.75.130:50217)
0923 15:02:25.213314 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0923 15:02:25.213326 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 15:02:25.213388 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:02:25.213721 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:02:25.213723 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:02:25.213895 (+ 172us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:02:25.213944 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:25.213946 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:25.214734 (+ 788us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:25.214735 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:25.215315 (+ 580us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:25.215317 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:25.215734 (+ 417us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:25.215735 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:25.215786 (+ 51us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:25.216296 (+ 510us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:02:25.216301 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:02:25.218223 (+ 1922us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.75.190:40153: Server oryx/127.13.75.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":89,"threads_started":1}
W20260923 15:02:25.218413 22807 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 15:02:25.213317 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.130:50217 (local address 127.13.75.190:40153)
0923 15:02:25.213459 (+ 142us) server_negotiation.cc:206] Beginning negotiation
0923 15:02:25.213461 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:02:25.213471 (+ 10us) server_negotiation.cc:407] Connection header received
0923 15:02:25.213503 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:02:25.213503 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:02:25.213514 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:02:25.213580 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:02:25.214079 (+ 499us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:25.214630 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:25.215413 (+ 783us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:25.215633 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:25.215713 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:25.218335 (+ 2622us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.75.130:50217: BlockingRecv error: recv got EOF from 127.13.75.130:50217 (error 108)
Metrics: {"server-negotiator.queue_time_us":109}
Sep 23 15:02:25 dist-test-slave-cnb0 krb5kdc[21755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.75.131@KRBTEST.COM for oryx/127.13.75.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 15:02:25 dist-test-slave-cnb0 krb5kdc[21755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.75.131@KRBTEST.COM for oryx/127.13.75.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 15:02:25.362480 22809 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:25.352732 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:40153 (local address 127.13.75.131:45659)
0923 15:02:25.352899 (+ 167us) negotiation.cc:107] Waiting for socket to connect
0923 15:02:25.352918 (+ 19us) client_negotiation.cc:174] Beginning negotiation
0923 15:02:25.353008 (+ 90us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:02:25.353171 (+ 163us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:02:25.353172 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:02:25.353341 (+ 169us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:02:25.353387 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:25.353389 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:25.354124 (+ 735us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:25.354126 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:25.354664 (+ 538us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:25.354666 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:25.355094 (+ 428us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:25.355095 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:25.355138 (+ 43us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:25.360250 (+ 5112us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:02:25.360257 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:02:25.362420 (+ 2163us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.75.190:40153: Server oryx/127.13.75.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":81,"threads_started":1}
W20260923 15:02:25.362637 22807 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 15:02:25.352852 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.131:45659 (local address 127.13.75.190:40153)
0923 15:02:25.352943 (+ 91us) server_negotiation.cc:206] Beginning negotiation
0923 15:02:25.352946 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:02:25.352973 (+ 27us) server_negotiation.cc:407] Connection header received
0923 15:02:25.353079 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:02:25.353080 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:02:25.353094 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:02:25.353157 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:02:25.353478 (+ 321us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:25.354010 (+ 532us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:25.354780 (+ 770us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:25.355020 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:25.355086 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:25.362538 (+ 7452us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.75.131:45659: BlockingRecv error: recv got EOF from 127.13.75.131:45659 (error 108)
Metrics: {"server-negotiator.queue_time_us":56}
Sep 23 15:02:26 dist-test-slave-cnb0 krb5kdc[21755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.75.129@KRBTEST.COM for oryx/127.13.75.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 15:02:26 dist-test-slave-cnb0 krb5kdc[21755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.75.129@KRBTEST.COM for oryx/127.13.75.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 15:02:26.097508 22810 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:26.088661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:40153 (local address 127.13.75.129:50635)
0923 15:02:26.088861 (+ 200us) negotiation.cc:107] Waiting for socket to connect
0923 15:02:26.088875 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0923 15:02:26.088970 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:02:26.089250 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:02:26.089251 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:02:26.089525 (+ 274us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:02:26.089585 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:26.089588 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:26.090328 (+ 740us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:26.090329 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:26.091112 (+ 783us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:26.091116 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:26.091572 (+ 456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:26.091573 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:26.091633 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:26.095454 (+ 3821us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:02:26.095461 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:02:26.097442 (+ 1981us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.75.190:40153: Server oryx/127.13.75.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":174,"thread_start_us":67,"threads_started":1}
W20260923 15:02:26.097757 22811 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 15:02:26.088912 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.129:50635 (local address 127.13.75.190:40153)
0923 15:02:26.089065 (+ 153us) server_negotiation.cc:206] Beginning negotiation
0923 15:02:26.089067 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:02:26.089076 (+ 9us) server_negotiation.cc:407] Connection header received
0923 15:02:26.089100 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:02:26.089101 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:02:26.089117 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:02:26.089194 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:02:26.089684 (+ 490us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:26.090225 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:26.091225 (+ 1000us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:26.091476 (+ 251us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:26.091548 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:26.097669 (+ 6121us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.75.129:50635: BlockingRecv error: recv got EOF from 127.13.75.129:50635 (error 108)
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":59,"threads_started":1}
Sep 23 15:02:26 dist-test-slave-cnb0 krb5kdc[21755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.75.130@KRBTEST.COM for oryx/127.13.75.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 15:02:26 dist-test-slave-cnb0 krb5kdc[21755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.75.130@KRBTEST.COM for oryx/127.13.75.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 15:02:26.224529 22812 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:26.219007 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:40153 (local address 127.13.75.130:33805)
0923 15:02:26.219203 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0923 15:02:26.219213 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 15:02:26.219272 (+ 59us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:02:26.219663 (+ 391us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:02:26.219665 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:02:26.219851 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:02:26.219903 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:26.219905 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:26.220598 (+ 693us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:26.220599 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:26.221194 (+ 595us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:26.221196 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:26.221621 (+ 425us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:26.221623 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:26.221686 (+ 63us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:26.222258 (+ 572us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:02:26.222262 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:02:26.224455 (+ 2193us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.75.190:40153: Server oryx/127.13.75.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":129,"threads_started":1}
W20260923 15:02:26.224601 22811 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 15:02:26.219108 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.130:33805 (local address 127.13.75.190:40153)
0923 15:02:26.219186 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0923 15:02:26.219188 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:02:26.219267 (+ 79us) server_negotiation.cc:407] Connection header received
0923 15:02:26.219508 (+ 241us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:02:26.219509 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:02:26.219525 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:02:26.219586 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:02:26.219987 (+ 401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:26.220517 (+ 530us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:26.221273 (+ 756us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:26.221484 (+ 211us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:26.221536 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:26.224551 (+ 3015us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.75.130:33805: BlockingRecv error: recv got EOF from 127.13.75.130:33805 (error 108)
Metrics: {"server-negotiator.queue_time_us":55}
Sep 23 15:02:26 dist-test-slave-cnb0 krb5kdc[21755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.75.131@KRBTEST.COM for oryx/127.13.75.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 15:02:26 dist-test-slave-cnb0 krb5kdc[21755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.75.131@KRBTEST.COM for oryx/127.13.75.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 15:02:26.368343 22813 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:26.363032 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:40153 (local address 127.13.75.131:55857)
0923 15:02:26.363317 (+ 285us) negotiation.cc:107] Waiting for socket to connect
0923 15:02:26.363329 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 15:02:26.363422 (+ 93us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:02:26.363591 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:02:26.363592 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:02:26.363777 (+ 185us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:02:26.363846 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:26.363848 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:26.364697 (+ 849us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:26.364698 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:26.365328 (+ 630us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:26.365330 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:26.365715 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:26.365716 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:26.365763 (+ 47us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:26.366258 (+ 495us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:02:26.366263 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:02:26.368230 (+ 1967us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.75.190:40153: Server oryx/127.13.75.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":266,"thread_start_us":98,"threads_started":1}
W20260923 15:02:26.368383 22811 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 15:02:26.363139 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.131:55857 (local address 127.13.75.190:40153)
0923 15:02:26.363312 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0923 15:02:26.363314 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:02:26.363401 (+ 87us) server_negotiation.cc:407] Connection header received
0923 15:02:26.363488 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:02:26.363489 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:02:26.363503 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:02:26.363584 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:02:26.363985 (+ 401us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:26.364582 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:26.365418 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:26.365639 (+ 221us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:26.365698 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:26.368316 (+ 2618us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.75.131:55857: BlockingRecv error: recv got EOF from 127.13.75.131:55857 (error 108)
Metrics: {"server-negotiator.queue_time_us":142}
Sep 23 15:02:27 dist-test-slave-cnb0 krb5kdc[21755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.75.129@KRBTEST.COM for oryx/127.13.75.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 15:02:27 dist-test-slave-cnb0 krb5kdc[21755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.75.129@KRBTEST.COM for oryx/127.13.75.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 15:02:27.103435 22814 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:27.098149 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:40153 (local address 127.13.75.129:59883)
0923 15:02:27.098343 (+ 194us) negotiation.cc:107] Waiting for socket to connect
0923 15:02:27.098354 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 15:02:27.098422 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:02:27.098659 (+ 237us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:02:27.098660 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:02:27.098833 (+ 173us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:02:27.098963 (+ 130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:27.098970 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:27.099721 (+ 751us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:27.099723 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:27.100354 (+ 631us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:27.100356 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:27.100830 (+ 474us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:27.100831 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:27.100923 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:27.101406 (+ 483us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:02:27.101411 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:02:27.103370 (+ 1959us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.75.190:40153: Server oryx/127.13.75.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":171,"thread_start_us":87,"threads_started":1}
W20260923 15:02:27.103626 22815 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 15:02:27.098289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.129:59883 (local address 127.13.75.190:40153)
0923 15:02:27.098468 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0923 15:02:27.098471 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:02:27.098480 (+ 9us) server_negotiation.cc:407] Connection header received
0923 15:02:27.098506 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:02:27.098507 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:02:27.098518 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:02:27.098581 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:02:27.099064 (+ 483us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:27.099610 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:27.100454 (+ 844us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:27.100672 (+ 218us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:27.100788 (+ 116us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:27.103546 (+ 2758us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.75.129:59883: BlockingRecv error: recv got EOF from 127.13.75.129:59883 (error 108)
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":121,"threads_started":1}
Sep 23 15:02:27 dist-test-slave-cnb0 krb5kdc[21755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.75.130@KRBTEST.COM for oryx/127.13.75.190@KRBTEST.COM, Server not found in Kerberos database
Sep 23 15:02:27 dist-test-slave-cnb0 krb5kdc[21755](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: LOOKING_UP_SERVER: authtime 0, oryx/127.13.75.130@KRBTEST.COM for oryx/127.13.75.190@KRBTEST.COM, Server not found in Kerberos database
I20260923 15:02:27.229820 22816 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:27.225113 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:40153 (local address 127.13.75.130:42905)
0923 15:02:27.225264 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0923 15:02:27.225276 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 15:02:27.225342 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:02:27.225478 (+ 136us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:02:27.225479 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:02:27.225654 (+ 175us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:02:27.225702 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:27.225704 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:27.226421 (+ 717us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:27.226422 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:27.226991 (+ 569us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:27.226992 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:27.227327 (+ 335us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:27.227328 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:27.227370 (+ 42us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:27.227847 (+ 477us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:02:27.227851 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:02:27.229759 (+ 1908us) negotiation.cc:327] Negotiation complete: Not authorized: Client connection negotiation failed: client connection to 127.13.75.190:40153: Server oryx/127.13.75.190@KRBTEST.COM not found in Kerberos database
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":92,"threads_started":1}
W20260923 15:02:27.229982 22815 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 15:02:27.225224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.130:42905 (local address 127.13.75.190:40153)
0923 15:02:27.225278 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0923 15:02:27.225279 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 15:02:27.225328 (+ 49us) server_negotiation.cc:407] Connection header received
0923 15:02:27.225402 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:02:27.225402 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:02:27.225414 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:02:27.225456 (+ 42us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:02:27.225804 (+ 348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:27.226335 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:27.227059 (+ 724us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:27.227255 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:27.227315 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:27.229908 (+ 2593us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.13.75.130:42905: BlockingRecv error: recv got EOF from 127.13.75.130:42905 (error 108)
Metrics: {"server-negotiator.queue_time_us":25}
I20260923 15:02:27.277519 13614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKC4soy/build/release/bin/kudu with pid 22339
I20260923 15:02:27.282258 13614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKC4soy/build/release/bin/kudu with pid 22491
I20260923 15:02:27.286686 13614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKC4soy/build/release/bin/kudu with pid 22624
I20260923 15:02:27.291455 13614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKC4soy/build/release/bin/kudu with pid 22269
2026-09-23T15:02:27Z chronyd exiting
[ OK ] SecurityITest.TestMismatchingPrincipals (17683 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationInsecureCluster
2026-09-23T15:02:27Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T15:02:27Z Disabled control of system clock
I20260923 15:02:27.320348 13614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKC4soy/build/release/bin/kudu
/tmp/dist-test-taskKC4soy/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790175691491353-13614-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790175691491353-13614-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790175691491353-13614-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790175691491353-13614-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.75.190:35469
--webserver_interface=127.13.75.190
--webserver_port=0
--builtin_ntp_servers=127.13.75.148:35769
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.75.190:35469
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {}
W20260923 15:02:27.409564 22820 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:02:27.409799 22820 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:02:27.409822 22820 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:02:27.411110 22820 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 15:02:27.411159 22820 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:02:27.411172 22820 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:02:27.411185 22820 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 15:02:27.411198 22820 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 15:02:27.412580 22820 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.75.148:35769
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790175691491353-13614-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790175691491353-13614-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.75.190:35469
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.75.190:35469
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790175691491353-13614-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.75.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22820
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790175691491353-13614-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:02:27.412858 22820 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:02:27.413192 22820 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:02:27.419251 22828 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:02:27.419302 22826 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:02:27.419284 22825 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:02:27.419351 22820 server_base.cc:1034] running on GCE node
I20260923 15:02:27.419775 22820 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:02:27.420125 22820 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:02:27.421286 22820 hybrid_clock.cc:648] HybridClock initialized: now 1790175747421270 us; error 30 us; skew 500 ppm
I20260923 15:02:27.422631 22820 webserver.cc:460] Webserver started at http://127.13.75.190:38057/ using document root <none> and password file <none>
I20260923 15:02:27.423022 22820 fs_manager.cc:359] Metadata directory not provided
I20260923 15:02:27.423080 22820 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:02:27.423236 22820 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:02:27.424407 22820 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790175691491353-13614-0/minicluster-data/master-0/data/instance:
uuid: "781548c5f1ac4c6cb1932d399f0e190a"
format_stamp: "Formatted at 2026-09-23 15:02:27 on dist-test-slave-cnb0"
server_key: "644cf59fd2d2c77b708f2928e3d02116"
server_key_iv: "b8daa7da1fc53b80d1eab748c68b7689"
server_key_version: "encryptionkey@0"
I20260923 15:02:27.424805 22820 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790175691491353-13614-0/minicluster-data/master-0/wal/instance:
uuid: "781548c5f1ac4c6cb1932d399f0e190a"
format_stamp: "Formatted at 2026-09-23 15:02:27 on dist-test-slave-cnb0"
server_key: "644cf59fd2d2c77b708f2928e3d02116"
server_key_iv: "b8daa7da1fc53b80d1eab748c68b7689"
server_key_version: "encryptionkey@0"
I20260923 15:02:27.426344 22820 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 15:02:27.427579 22834 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:02:27.427855 22820 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260923 15:02:27.427973 22820 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790175691491353-13614-0/minicluster-data/master-0/data,/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790175691491353-13614-0/minicluster-data/master-0/wal
uuid: "781548c5f1ac4c6cb1932d399f0e190a"
format_stamp: "Formatted at 2026-09-23 15:02:27 on dist-test-slave-cnb0"
server_key: "644cf59fd2d2c77b708f2928e3d02116"
server_key_iv: "b8daa7da1fc53b80d1eab748c68b7689"
server_key_version: "encryptionkey@0"
I20260923 15:02:27.428098 22820 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790175691491353-13614-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790175691491353-13614-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790175691491353-13614-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:02:27.439834 22820 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:02:27.440212 22820 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:02:27.440395 22820 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:02:27.445196 22820 rpc_server.cc:307] RPC server started. Bound to: 127.13.75.190:35469
I20260923 15:02:27.445259 22886 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.75.190:35469 every 8 connection(s)
I20260923 15:02:27.445681 22820 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790175691491353-13614-0/minicluster-data/master-0/data/info.pb
I20260923 15:02:27.446467 22887 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:02:27.449333 22887 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 781548c5f1ac4c6cb1932d399f0e190a: Bootstrap starting.
I20260923 15:02:27.449970 22887 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 781548c5f1ac4c6cb1932d399f0e190a: Neither blocks nor log segments found. Creating new log.
I20260923 15:02:27.450289 22887 log.cc:824] T 00000000000000000000000000000000 P 781548c5f1ac4c6cb1932d399f0e190a: Log is configured to *not* fsync() on all Append() calls
I20260923 15:02:27.451188 22887 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 781548c5f1ac4c6cb1932d399f0e190a: No bootstrap required, opened a new log
I20260923 15:02:27.453163 22887 raft_consensus.cc:348] T 00000000000000000000000000000000 P 781548c5f1ac4c6cb1932d399f0e190a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "781548c5f1ac4c6cb1932d399f0e190a" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 35469 } }
I20260923 15:02:27.453339 22887 raft_consensus.cc:374] T 00000000000000000000000000000000 P 781548c5f1ac4c6cb1932d399f0e190a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:02:27.453526 22887 raft_consensus.cc:729] T 00000000000000000000000000000000 P 781548c5f1ac4c6cb1932d399f0e190a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 781548c5f1ac4c6cb1932d399f0e190a, State: Initialized, Role: FOLLOWER
I20260923 15:02:27.453745 22887 consensus_queue.cc:260] T 00000000000000000000000000000000 P 781548c5f1ac4c6cb1932d399f0e190a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "781548c5f1ac4c6cb1932d399f0e190a" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 35469 } }
I20260923 15:02:27.453890 22887 raft_consensus.cc:388] T 00000000000000000000000000000000 P 781548c5f1ac4c6cb1932d399f0e190a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 15:02:27.453958 22887 raft_consensus.cc:482] T 00000000000000000000000000000000 P 781548c5f1ac4c6cb1932d399f0e190a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 15:02:27.454039 22887 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 781548c5f1ac4c6cb1932d399f0e190a [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:02:27.455020 22887 raft_consensus.cc:504] T 00000000000000000000000000000000 P 781548c5f1ac4c6cb1932d399f0e190a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "781548c5f1ac4c6cb1932d399f0e190a" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 35469 } }
I20260923 15:02:27.455120 22887 leader_election.cc:302] T 00000000000000000000000000000000 P 781548c5f1ac4c6cb1932d399f0e190a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 781548c5f1ac4c6cb1932d399f0e190a; no voters:
I20260923 15:02:27.455322 22887 leader_election.cc:288] T 00000000000000000000000000000000 P 781548c5f1ac4c6cb1932d399f0e190a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 15:02:27.455436 22890 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 781548c5f1ac4c6cb1932d399f0e190a [term 1 FOLLOWER]: Leader election won for term 1
I20260923 15:02:27.455605 22890 raft_consensus.cc:686] T 00000000000000000000000000000000 P 781548c5f1ac4c6cb1932d399f0e190a [term 1 LEADER]: Becoming Leader. State: Replica: 781548c5f1ac4c6cb1932d399f0e190a, State: Running, Role: LEADER
I20260923 15:02:27.455608 22887 sys_catalog.cc:564] T 00000000000000000000000000000000 P 781548c5f1ac4c6cb1932d399f0e190a [sys.catalog]: configured and running, proceeding with master startup.
I20260923 15:02:27.455855 13614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKC4soy/build/release/bin/kudu as pid 22820
I20260923 15:02:27.455761 22890 consensus_queue.cc:237] T 00000000000000000000000000000000 P 781548c5f1ac4c6cb1932d399f0e190a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "781548c5f1ac4c6cb1932d399f0e190a" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 35469 } }
I20260923 15:02:27.455930 13614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790175691491353-13614-0/minicluster-data/master-0/wal/instance
I20260923 15:02:27.456147 13614 external_mini_cluster.cc:1428] Setting key 4e66dfb5f8f8ed515aa50302c9fa0b3c
I20260923 15:02:27.456450 22892 sys_catalog.cc:455] T 00000000000000000000000000000000 P 781548c5f1ac4c6cb1932d399f0e190a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 781548c5f1ac4c6cb1932d399f0e190a. Latest consensus state: current_term: 1 leader_uuid: "781548c5f1ac4c6cb1932d399f0e190a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "781548c5f1ac4c6cb1932d399f0e190a" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 35469 } } }
I20260923 15:02:27.456612 22892 sys_catalog.cc:458] T 00000000000000000000000000000000 P 781548c5f1ac4c6cb1932d399f0e190a [sys.catalog]: This master's current role is: LEADER
I20260923 15:02:27.456593 22891 sys_catalog.cc:455] T 00000000000000000000000000000000 P 781548c5f1ac4c6cb1932d399f0e190a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "781548c5f1ac4c6cb1932d399f0e190a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "781548c5f1ac4c6cb1932d399f0e190a" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 35469 } } }
I20260923 15:02:27.456688 22891 sys_catalog.cc:458] T 00000000000000000000000000000000 P 781548c5f1ac4c6cb1932d399f0e190a [sys.catalog]: This master's current role is: LEADER
W20260923 15:02:27.458236 22908 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 781548c5f1ac4c6cb1932d399f0e190a: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260923 15:02:27.458346 22908 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260923 15:02:27.458459 22896 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 15:02:27.458681 22896 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 15:02:27.460431 22896 catalog_manager.cc:1347] Generated new cluster ID: 0ea8f7d044b7466888858721690b357c
I20260923 15:02:27.460492 22896 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 15:02:27.467794 22896 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 15:02:27.468560 22896 catalog_manager.cc:1504] Loading token signing keys...
I20260923 15:02:27.470499 22897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:27.456666 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55606 (local address 127.13.75.190:35469)
0923 15:02:27.458096 (+ 1430us) server_negotiation.cc:206] Beginning negotiation
0923 15:02:27.458105 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0923 15:02:27.458131 (+ 26us) server_negotiation.cc:407] Connection header received
0923 15:02:27.458237 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:02:27.458243 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:02:27.458301 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:02:27.459295 (+ 994us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:02:27.459386 (+ 91us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:27.467183 (+ 7797us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:27.468228 (+ 1045us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:27.468774 (+ 546us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:27.468927 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:27.469685 (+ 758us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:02:27.469689 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:02:27.469689 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 15:02:27.469883 (+ 194us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:02:27.469922 (+ 39us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 15:02:27.469934 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:02:27.470018 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0923 15:02:27.470102 (+ 84us) server_negotiation.cc:299] Negotiation successful
0923 15:02:27.470310 (+ 208us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1364,"thread_start_us":78,"threads_started":1}
I20260923 15:02:27.496723 22896 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 781548c5f1ac4c6cb1932d399f0e190a: Generated new TSK 0
I20260923 15:02:27.497000 22896 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 15:02:27.508924 13614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKC4soy/build/release/bin/kudu
/tmp/dist-test-taskKC4soy/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790175691491353-13614-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790175691491353-13614-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790175691491353-13614-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790175691491353-13614-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.75.129:0
--local_ip_for_outbound_sockets=127.13.75.129
--webserver_interface=127.13.75.129
--webserver_port=0
--tserver_master_addrs=127.13.75.190:35469
--builtin_ntp_servers=127.13.75.148:35769
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260923 15:02:27.611671 22911 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:02:27.611887 22911 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:02:27.611912 22911 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:02:27.613231 22911 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:02:27.613276 22911 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:02:27.613303 22911 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.75.129
I20260923 15:02:27.614768 22911 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.75.148:35769
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790175691491353-13614-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790175691491353-13614-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.75.129:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790175691491353-13614-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.75.129
--webserver_port=0
--tserver_master_addrs=127.13.75.190:35469
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.22911
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.75.129
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790175691491353-13614-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:02:27.615059 22911 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:02:27.615373 22911 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:02:27.621471 22917 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:02:27.621491 22919 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:02:27.621492 22916 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:02:27.621949 22911 server_base.cc:1034] running on GCE node
I20260923 15:02:27.622210 22911 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:02:27.622503 22911 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:02:27.623724 22911 hybrid_clock.cc:648] HybridClock initialized: now 1790175747623711 us; error 40 us; skew 500 ppm
I20260923 15:02:27.625206 22911 webserver.cc:460] Webserver started at http://127.13.75.129:34629/ using document root <none> and password file <none>
I20260923 15:02:27.625468 22911 fs_manager.cc:359] Metadata directory not provided
I20260923 15:02:27.625591 22911 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:02:27.625761 22911 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:02:27.627075 22911 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790175691491353-13614-0/minicluster-data/ts-0/data/instance:
uuid: "45bc395df902411cbb7b1cd19bc0d320"
format_stamp: "Formatted at 2026-09-23 15:02:27 on dist-test-slave-cnb0"
server_key: "3c54e8877cd4bd0131b526a7e11843c6"
server_key_iv: "20d268212235fad635e31a55bebb7b96"
server_key_version: "encryptionkey@0"
I20260923 15:02:27.627476 22911 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790175691491353-13614-0/minicluster-data/ts-0/wal/instance:
uuid: "45bc395df902411cbb7b1cd19bc0d320"
format_stamp: "Formatted at 2026-09-23 15:02:27 on dist-test-slave-cnb0"
server_key: "3c54e8877cd4bd0131b526a7e11843c6"
server_key_iv: "20d268212235fad635e31a55bebb7b96"
server_key_version: "encryptionkey@0"
I20260923 15:02:27.628952 22911 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260923 15:02:27.629912 22925 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:02:27.630156 22911 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260923 15:02:27.630268 22911 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790175691491353-13614-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790175691491353-13614-0/minicluster-data/ts-0/wal
uuid: "45bc395df902411cbb7b1cd19bc0d320"
format_stamp: "Formatted at 2026-09-23 15:02:27 on dist-test-slave-cnb0"
server_key: "3c54e8877cd4bd0131b526a7e11843c6"
server_key_iv: "20d268212235fad635e31a55bebb7b96"
server_key_version: "encryptionkey@0"
I20260923 15:02:27.630352 22911 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790175691491353-13614-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790175691491353-13614-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790175691491353-13614-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:02:27.648113 22911 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:02:27.648506 22911 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:02:27.648681 22911 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:02:27.648936 22911 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 15:02:27.649281 22911 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 15:02:27.649338 22911 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 15:02:27.649381 22911 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 15:02:27.649406 22911 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 15:02:27.655844 22911 rpc_server.cc:307] RPC server started. Bound to: 127.13.75.129:39959
I20260923 15:02:27.655920 23038 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.75.129:39959 every 8 connection(s)
I20260923 15:02:27.656297 22911 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790175691491353-13614-0/minicluster-data/ts-0/data/info.pb
I20260923 15:02:27.663738 22897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:27.657231 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.129:45065 (local address 127.13.75.190:35469)
0923 15:02:27.657299 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0923 15:02:27.657301 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:02:27.657492 (+ 191us) server_negotiation.cc:407] Connection header received
0923 15:02:27.658132 (+ 640us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:02:27.658134 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:02:27.658148 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:02:27.658219 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:02:27.658988 (+ 769us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:27.659861 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:27.661708 (+ 1847us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:27.662071 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:27.662158 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:27.663311 (+ 1153us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:02:27.663312 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:02:27.663313 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 15:02:27.663320 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:02:27.663340 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 15:02:27.663345 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:02:27.663407 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0923 15:02:27.663589 (+ 182us) server_negotiation.cc:299] Negotiation successful
0923 15:02:27.663639 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260923 15:02:27.664002 23041 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:27.657157 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:35469 (local address 127.13.75.129:45065)
0923 15:02:27.657346 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0923 15:02:27.657391 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0923 15:02:27.657974 (+ 583us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:02:27.658278 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:02:27.658282 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:02:27.658290 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:02:27.658805 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:27.658810 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:27.660036 (+ 1226us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:27.660037 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:27.661587 (+ 1550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:27.661590 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:27.662188 (+ 598us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:27.662189 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:27.662395 (+ 206us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:27.663115 (+ 720us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 15:02:27.663115 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:02:27.663160 (+ 45us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 15:02:27.663192 (+ 32us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:02:27.663450 (+ 258us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:02:27.663463 (+ 13us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:02:27.663466 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 15:02:27.663558 (+ 92us) client_negotiation.cc:240] Negotiation successful
0923 15:02:27.663709 (+ 151us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":90,"threads_started":1}
I20260923 15:02:27.664999 23039 heartbeater.cc:347] Connected to a master server at 127.13.75.190:35469
I20260923 15:02:27.665133 23039 heartbeater.cc:464] Registering TS with master...
I20260923 15:02:27.665386 23039 heartbeater.cc:511] Master 127.13.75.190:35469 requested a full tablet report, sending...
I20260923 15:02:27.665503 13614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKC4soy/build/release/bin/kudu as pid 22911
I20260923 15:02:27.665644 13614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790175691491353-13614-0/minicluster-data/ts-0/wal/instance
I20260923 15:02:27.665863 13614 external_mini_cluster.cc:1428] Setting key 167ec2ad56fe972b1b9f0c8dcb3269ec
I20260923 15:02:27.666504 22851 ts_manager.cc:194] Registered new tserver with Master: 45bc395df902411cbb7b1cd19bc0d320 (127.13.75.129:39959)
I20260923 15:02:27.667232 13614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKC4soy/build/release/bin/kudu
/tmp/dist-test-taskKC4soy/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790175691491353-13614-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790175691491353-13614-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790175691491353-13614-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790175691491353-13614-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.75.130:0
--local_ip_for_outbound_sockets=127.13.75.130
--webserver_interface=127.13.75.130
--webserver_port=0
--tserver_master_addrs=127.13.75.190:35469
--builtin_ntp_servers=127.13.75.148:35769
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
I20260923 15:02:27.667294 22851 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.13.75.129:45065
W20260923 15:02:27.751888 23042 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:02:27.752152 23042 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:02:27.752192 23042 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:02:27.753482 23042 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:02:27.753561 23042 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:02:27.753603 23042 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.75.130
I20260923 15:02:27.755065 23042 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.75.148:35769
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790175691491353-13614-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790175691491353-13614-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.75.130:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790175691491353-13614-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.75.130
--webserver_port=0
--tserver_master_addrs=127.13.75.190:35469
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23042
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.75.130
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790175691491353-13614-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:02:27.755398 23042 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:02:27.755779 23042 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:02:27.761185 23047 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:02:27.761245 23048 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:02:27.761277 23050 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:02:27.761461 23042 server_base.cc:1034] running on GCE node
I20260923 15:02:27.761780 23042 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:02:27.762066 23042 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:02:27.763239 23042 hybrid_clock.cc:648] HybridClock initialized: now 1790175747763222 us; error 28 us; skew 500 ppm
I20260923 15:02:27.764616 23042 webserver.cc:460] Webserver started at http://127.13.75.130:33453/ using document root <none> and password file <none>
I20260923 15:02:27.764914 23042 fs_manager.cc:359] Metadata directory not provided
I20260923 15:02:27.764990 23042 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:02:27.765168 23042 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:02:27.766326 23042 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790175691491353-13614-0/minicluster-data/ts-1/data/instance:
uuid: "27351cf6db9a45d08bf67c14905c8603"
format_stamp: "Formatted at 2026-09-23 15:02:27 on dist-test-slave-cnb0"
server_key: "258aef9aee318f6662c986140f950ed0"
server_key_iv: "5ec83340810fdc817477df58ad5016bb"
server_key_version: "encryptionkey@0"
I20260923 15:02:27.766738 23042 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790175691491353-13614-0/minicluster-data/ts-1/wal/instance:
uuid: "27351cf6db9a45d08bf67c14905c8603"
format_stamp: "Formatted at 2026-09-23 15:02:27 on dist-test-slave-cnb0"
server_key: "258aef9aee318f6662c986140f950ed0"
server_key_iv: "5ec83340810fdc817477df58ad5016bb"
server_key_version: "encryptionkey@0"
I20260923 15:02:27.768420 23042 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260923 15:02:27.769369 23056 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:02:27.769554 23042 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260923 15:02:27.769655 23042 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790175691491353-13614-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790175691491353-13614-0/minicluster-data/ts-1/wal
uuid: "27351cf6db9a45d08bf67c14905c8603"
format_stamp: "Formatted at 2026-09-23 15:02:27 on dist-test-slave-cnb0"
server_key: "258aef9aee318f6662c986140f950ed0"
server_key_iv: "5ec83340810fdc817477df58ad5016bb"
server_key_version: "encryptionkey@0"
I20260923 15:02:27.769740 23042 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790175691491353-13614-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790175691491353-13614-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790175691491353-13614-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:02:27.795442 23042 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:02:27.795785 23042 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:02:27.795944 23042 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:02:27.796213 23042 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 15:02:27.796573 23042 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 15:02:27.796628 23042 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 15:02:27.796664 23042 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 15:02:27.796692 23042 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 15:02:27.803309 23042 rpc_server.cc:307] RPC server started. Bound to: 127.13.75.130:42827
I20260923 15:02:27.803354 23169 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.75.130:42827 every 8 connection(s)
I20260923 15:02:27.803719 23042 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790175691491353-13614-0/minicluster-data/ts-1/data/info.pb
I20260923 15:02:27.809334 22897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:27.804498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.130:48191 (local address 127.13.75.190:35469)
0923 15:02:27.804600 (+ 102us) server_negotiation.cc:206] Beginning negotiation
0923 15:02:27.804602 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:02:27.804830 (+ 228us) server_negotiation.cc:407] Connection header received
0923 15:02:27.805270 (+ 440us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:02:27.805271 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:02:27.805283 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:02:27.805352 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:02:27.806128 (+ 776us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:27.806631 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:27.807765 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:27.807995 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:27.808062 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:27.808939 (+ 877us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:02:27.808940 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:02:27.808940 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 15:02:27.808946 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:02:27.808962 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 15:02:27.808966 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:02:27.809024 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0923 15:02:27.809217 (+ 193us) server_negotiation.cc:299] Negotiation successful
0923 15:02:27.809261 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":70}
I20260923 15:02:27.809499 23172 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:27.804435 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:35469 (local address 127.13.75.130:48191)
0923 15:02:27.804640 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0923 15:02:27.804683 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0923 15:02:27.805170 (+ 487us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:02:27.805503 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:02:27.805508 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:02:27.805517 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:02:27.806036 (+ 519us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:27.806041 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:27.806741 (+ 700us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:27.806742 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:27.807667 (+ 925us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:27.807670 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:27.808081 (+ 411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:27.808082 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:27.808198 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:27.808738 (+ 540us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 15:02:27.808738 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:02:27.808779 (+ 41us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 15:02:27.808805 (+ 26us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:02:27.809077 (+ 272us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:02:27.809083 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:02:27.809086 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 15:02:27.809172 (+ 86us) client_negotiation.cc:240] Negotiation successful
0923 15:02:27.809275 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":97,"threads_started":1}
I20260923 15:02:27.810094 23170 heartbeater.cc:347] Connected to a master server at 127.13.75.190:35469
I20260923 15:02:27.810209 23170 heartbeater.cc:464] Registering TS with master...
I20260923 15:02:27.810397 23170 heartbeater.cc:511] Master 127.13.75.190:35469 requested a full tablet report, sending...
I20260923 15:02:27.811048 22851 ts_manager.cc:194] Registered new tserver with Master: 27351cf6db9a45d08bf67c14905c8603 (127.13.75.130:42827)
I20260923 15:02:27.811556 22851 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.13.75.130:48191
I20260923 15:02:27.812891 13614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKC4soy/build/release/bin/kudu as pid 23042
I20260923 15:02:27.812994 13614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790175691491353-13614-0/minicluster-data/ts-1/wal/instance
I20260923 15:02:27.813156 13614 external_mini_cluster.cc:1428] Setting key 0fa0c5b0c41ba54c48e3ac3e25bf24fa
I20260923 15:02:27.814215 13614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKC4soy/build/release/bin/kudu
/tmp/dist-test-taskKC4soy/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790175691491353-13614-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790175691491353-13614-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790175691491353-13614-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790175691491353-13614-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.75.131:0
--local_ip_for_outbound_sockets=127.13.75.131
--webserver_interface=127.13.75.131
--webserver_port=0
--tserver_master_addrs=127.13.75.190:35469
--builtin_ntp_servers=127.13.75.148:35769
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {}
W20260923 15:02:27.908034 23173 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:02:27.908254 23173 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:02:27.908277 23173 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:02:27.909461 23173 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:02:27.909502 23173 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:02:27.909526 23173 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.75.131
I20260923 15:02:27.910831 23173 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.75.148:35769
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790175691491353-13614-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790175691491353-13614-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.75.131:0
--rpc_server_allow_ephemeral_ports=true
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790175691491353-13614-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.75.131
--webserver_port=0
--tserver_master_addrs=127.13.75.190:35469
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23173
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.75.131
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790175691491353-13614-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:02:27.911123 23173 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:02:27.911403 23173 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:02:27.916445 23181 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:02:27.916674 23179 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:02:27.916760 23173 server_base.cc:1034] running on GCE node
W20260923 15:02:27.916790 23178 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:02:27.917155 23173 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:02:27.917367 23173 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:02:27.918509 23173 hybrid_clock.cc:648] HybridClock initialized: now 1790175747918493 us; error 27 us; skew 500 ppm
I20260923 15:02:27.919924 23173 webserver.cc:460] Webserver started at http://127.13.75.131:40873/ using document root <none> and password file <none>
I20260923 15:02:27.920176 23173 fs_manager.cc:359] Metadata directory not provided
I20260923 15:02:27.920225 23173 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:02:27.920378 23173 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:02:27.921474 23173 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790175691491353-13614-0/minicluster-data/ts-2/data/instance:
uuid: "8b8494cd3cbc47a1ad1078a797b16052"
format_stamp: "Formatted at 2026-09-23 15:02:27 on dist-test-slave-cnb0"
server_key: "befee1720335fdd03518e448ed114e48"
server_key_iv: "a07497e2490f2b5003aab7f34ae0fb33"
server_key_version: "encryptionkey@0"
I20260923 15:02:27.921857 23173 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790175691491353-13614-0/minicluster-data/ts-2/wal/instance:
uuid: "8b8494cd3cbc47a1ad1078a797b16052"
format_stamp: "Formatted at 2026-09-23 15:02:27 on dist-test-slave-cnb0"
server_key: "befee1720335fdd03518e448ed114e48"
server_key_iv: "a07497e2490f2b5003aab7f34ae0fb33"
server_key_version: "encryptionkey@0"
I20260923 15:02:27.923352 23173 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.002s
I20260923 15:02:27.924362 23187 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:02:27.924682 23173 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:02:27.924772 23173 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790175691491353-13614-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790175691491353-13614-0/minicluster-data/ts-2/wal
uuid: "8b8494cd3cbc47a1ad1078a797b16052"
format_stamp: "Formatted at 2026-09-23 15:02:27 on dist-test-slave-cnb0"
server_key: "befee1720335fdd03518e448ed114e48"
server_key_iv: "a07497e2490f2b5003aab7f34ae0fb33"
server_key_version: "encryptionkey@0"
I20260923 15:02:27.924829 23173 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790175691491353-13614-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790175691491353-13614-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790175691491353-13614-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:02:27.945760 23173 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:02:27.946113 23173 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:02:27.946271 23173 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:02:27.946547 23173 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 15:02:27.946969 23173 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 15:02:27.947032 23173 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 15:02:27.947070 23173 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 15:02:27.947099 23173 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 15:02:27.953474 23173 rpc_server.cc:307] RPC server started. Bound to: 127.13.75.131:38583
I20260923 15:02:27.953503 23300 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.75.131:38583 every 8 connection(s)
I20260923 15:02:27.953873 23173 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790175691491353-13614-0/minicluster-data/ts-2/data/info.pb
I20260923 15:02:27.960098 22897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:27.954697 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.131:53749 (local address 127.13.75.190:35469)
0923 15:02:27.954763 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0923 15:02:27.954765 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:02:27.955020 (+ 255us) server_negotiation.cc:407] Connection header received
0923 15:02:27.955562 (+ 542us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:02:27.955563 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:02:27.955578 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:02:27.955641 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:02:27.956399 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:27.957154 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:27.958336 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:27.958536 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:27.958591 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:27.959780 (+ 1189us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:02:27.959781 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:02:27.959781 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 15:02:27.959786 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:02:27.959802 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 15:02:27.959806 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:02:27.959896 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0923 15:02:27.959969 (+ 73us) server_negotiation.cc:299] Negotiation successful
0923 15:02:27.959998 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260923 15:02:27.960306 13614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKC4soy/build/release/bin/kudu as pid 23173
I20260923 15:02:27.960378 23303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:27.954663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:35469 (local address 127.13.75.131:53749)
0923 15:02:27.954871 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0923 15:02:27.954938 (+ 67us) client_negotiation.cc:174] Beginning negotiation
0923 15:02:27.955474 (+ 536us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:02:27.955698 (+ 224us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:02:27.955701 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:02:27.955710 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:02:27.956320 (+ 610us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:27.956327 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:27.957228 (+ 901us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:27.957229 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:27.958232 (+ 1003us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:27.958235 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:27.958634 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:27.958635 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:27.958779 (+ 144us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:27.959622 (+ 843us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 15:02:27.959622 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:02:27.959652 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 15:02:27.959676 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:02:27.959899 (+ 223us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:02:27.959902 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:02:27.959904 (+ 2us) client_negotiation.cc:769] Sending connection context
0923 15:02:27.959956 (+ 52us) client_negotiation.cc:240] Negotiation successful
0923 15:02:27.960067 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":88,"threads_started":1}
I20260923 15:02:27.960459 13614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationInsecureCluster.1790175691491353-13614-0/minicluster-data/ts-2/wal/instance
I20260923 15:02:27.960654 13614 external_mini_cluster.cc:1428] Setting key 94d4cb58291fd7fa1f32ce62c73b6462
I20260923 15:02:27.960973 23301 heartbeater.cc:347] Connected to a master server at 127.13.75.190:35469
I20260923 15:02:27.961079 23301 heartbeater.cc:464] Registering TS with master...
I20260923 15:02:27.961288 23301 heartbeater.cc:511] Master 127.13.75.190:35469 requested a full tablet report, sending...
I20260923 15:02:27.961733 22851 ts_manager.cc:194] Registered new tserver with Master: 8b8494cd3cbc47a1ad1078a797b16052 (127.13.75.131:38583)
I20260923 15:02:27.962119 22851 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.13.75.131:53749
I20260923 15:02:27.972675 13614 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
W20260923 15:02:27.974874 22897 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 15:02:27.974189 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55614 (local address 127.13.75.190:35469)
0923 15:02:27.974256 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0923 15:02:27.974258 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:02:27.974442 (+ 184us) server_negotiation.cc:407] Connection header received
0923 15:02:27.974470 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:02:27.974471 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:02:27.974481 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:02:27.974530 (+ 49us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:02:27.974774 (+ 244us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:55614: BlockingRecv error: recv got EOF from 127.0.0.1:55614 (error 108)
Metrics: {"server-negotiator.queue_time_us":40}
I20260923 15:02:27.974926 13614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKC4soy/build/release/bin/kudu with pid 22911
I20260923 15:02:27.979974 13614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKC4soy/build/release/bin/kudu with pid 23042
I20260923 15:02:27.984725 13614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKC4soy/build/release/bin/kudu with pid 23173
I20260923 15:02:27.989030 13614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKC4soy/build/release/bin/kudu with pid 22820
2026-09-23T15:02:27Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationInsecureCluster (696 ms)
[ RUN ] SecurityITest.TestRequireEncryptionInsecureCluster
2026-09-23T15:02:28Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T15:02:28Z Disabled control of system clock
I20260923 15:02:28.018013 13614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKC4soy/build/release/bin/kudu
/tmp/dist-test-taskKC4soy/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790175691491353-13614-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790175691491353-13614-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790175691491353-13614-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790175691491353-13614-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.75.190:37605
--webserver_interface=127.13.75.190
--webserver_port=0
--builtin_ntp_servers=127.13.75.148:37807
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.75.190:37605
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260923 15:02:28.106037 23312 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:02:28.106297 23312 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:02:28.106323 23312 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:02:28.107728 23312 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 15:02:28.107785 23312 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:02:28.107800 23312 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:02:28.107815 23312 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 15:02:28.107828 23312 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 15:02:28.109440 23312 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.75.148:37807
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790175691491353-13614-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790175691491353-13614-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.75.190:37605
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.75.190:37605
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790175691491353-13614-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.75.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23312
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790175691491353-13614-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:02:28.109756 23312 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:02:28.110112 23312 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:02:28.115960 23317 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:02:28.115953 23320 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:02:28.115953 23318 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:02:28.115985 23312 server_base.cc:1034] running on GCE node
I20260923 15:02:28.116506 23312 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:02:28.116791 23312 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:02:28.117942 23312 hybrid_clock.cc:648] HybridClock initialized: now 1790175748117929 us; error 34 us; skew 500 ppm
I20260923 15:02:28.119361 23312 webserver.cc:460] Webserver started at http://127.13.75.190:44715/ using document root <none> and password file <none>
I20260923 15:02:28.119640 23312 fs_manager.cc:359] Metadata directory not provided
I20260923 15:02:28.119695 23312 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:02:28.119836 23312 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:02:28.120995 23312 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790175691491353-13614-0/minicluster-data/master-0/data/instance:
uuid: "34612c2dd56544e48799380724ad75ef"
format_stamp: "Formatted at 2026-09-23 15:02:28 on dist-test-slave-cnb0"
server_key: "e94d9a7144cea34c5fa5bbd8966a4d06"
server_key_iv: "7a64cf1d5cd4dda39f0899618bf91a88"
server_key_version: "encryptionkey@0"
I20260923 15:02:28.121384 23312 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790175691491353-13614-0/minicluster-data/master-0/wal/instance:
uuid: "34612c2dd56544e48799380724ad75ef"
format_stamp: "Formatted at 2026-09-23 15:02:28 on dist-test-slave-cnb0"
server_key: "e94d9a7144cea34c5fa5bbd8966a4d06"
server_key_iv: "7a64cf1d5cd4dda39f0899618bf91a88"
server_key_version: "encryptionkey@0"
I20260923 15:02:28.122876 23312 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 15:02:28.123940 23326 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:02:28.124197 23312 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:02:28.124310 23312 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790175691491353-13614-0/minicluster-data/master-0/data,/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790175691491353-13614-0/minicluster-data/master-0/wal
uuid: "34612c2dd56544e48799380724ad75ef"
format_stamp: "Formatted at 2026-09-23 15:02:28 on dist-test-slave-cnb0"
server_key: "e94d9a7144cea34c5fa5bbd8966a4d06"
server_key_iv: "7a64cf1d5cd4dda39f0899618bf91a88"
server_key_version: "encryptionkey@0"
I20260923 15:02:28.124451 23312 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790175691491353-13614-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790175691491353-13614-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790175691491353-13614-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:02:28.128181 23312 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:02:28.128501 23312 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:02:28.128654 23312 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:02:28.132805 23312 rpc_server.cc:307] RPC server started. Bound to: 127.13.75.190:37605
I20260923 15:02:28.132887 23378 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.75.190:37605 every 8 connection(s)
I20260923 15:02:28.133255 23312 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790175691491353-13614-0/minicluster-data/master-0/data/info.pb
I20260923 15:02:28.134059 23379 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:02:28.137425 23379 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 34612c2dd56544e48799380724ad75ef: Bootstrap starting.
I20260923 15:02:28.138063 23379 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 34612c2dd56544e48799380724ad75ef: Neither blocks nor log segments found. Creating new log.
I20260923 15:02:28.138381 23379 log.cc:824] T 00000000000000000000000000000000 P 34612c2dd56544e48799380724ad75ef: Log is configured to *not* fsync() on all Append() calls
I20260923 15:02:28.139235 23379 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 34612c2dd56544e48799380724ad75ef: No bootstrap required, opened a new log
I20260923 15:02:28.140969 23379 raft_consensus.cc:348] T 00000000000000000000000000000000 P 34612c2dd56544e48799380724ad75ef [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34612c2dd56544e48799380724ad75ef" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 37605 } }
I20260923 15:02:28.141116 23379 raft_consensus.cc:374] T 00000000000000000000000000000000 P 34612c2dd56544e48799380724ad75ef [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:02:28.141139 23379 raft_consensus.cc:729] T 00000000000000000000000000000000 P 34612c2dd56544e48799380724ad75ef [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 34612c2dd56544e48799380724ad75ef, State: Initialized, Role: FOLLOWER
I20260923 15:02:28.141290 23379 consensus_queue.cc:260] T 00000000000000000000000000000000 P 34612c2dd56544e48799380724ad75ef [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34612c2dd56544e48799380724ad75ef" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 37605 } }
I20260923 15:02:28.141369 23379 raft_consensus.cc:388] T 00000000000000000000000000000000 P 34612c2dd56544e48799380724ad75ef [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 15:02:28.141395 23379 raft_consensus.cc:482] T 00000000000000000000000000000000 P 34612c2dd56544e48799380724ad75ef [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 15:02:28.141448 23379 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 34612c2dd56544e48799380724ad75ef [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:02:28.142092 23379 raft_consensus.cc:504] T 00000000000000000000000000000000 P 34612c2dd56544e48799380724ad75ef [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34612c2dd56544e48799380724ad75ef" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 37605 } }
I20260923 15:02:28.142204 23379 leader_election.cc:302] T 00000000000000000000000000000000 P 34612c2dd56544e48799380724ad75ef [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 34612c2dd56544e48799380724ad75ef; no voters:
I20260923 15:02:28.142408 23379 leader_election.cc:288] T 00000000000000000000000000000000 P 34612c2dd56544e48799380724ad75ef [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 15:02:28.142593 23382 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 34612c2dd56544e48799380724ad75ef [term 1 FOLLOWER]: Leader election won for term 1
I20260923 15:02:28.142716 23379 sys_catalog.cc:564] T 00000000000000000000000000000000 P 34612c2dd56544e48799380724ad75ef [sys.catalog]: configured and running, proceeding with master startup.
I20260923 15:02:28.142786 23382 raft_consensus.cc:686] T 00000000000000000000000000000000 P 34612c2dd56544e48799380724ad75ef [term 1 LEADER]: Becoming Leader. State: Replica: 34612c2dd56544e48799380724ad75ef, State: Running, Role: LEADER
I20260923 15:02:28.142920 23382 consensus_queue.cc:237] T 00000000000000000000000000000000 P 34612c2dd56544e48799380724ad75ef [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34612c2dd56544e48799380724ad75ef" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 37605 } }
I20260923 15:02:28.143411 13614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKC4soy/build/release/bin/kudu as pid 23312
I20260923 15:02:28.143536 13614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790175691491353-13614-0/minicluster-data/master-0/wal/instance
I20260923 15:02:28.143602 23383 sys_catalog.cc:455] T 00000000000000000000000000000000 P 34612c2dd56544e48799380724ad75ef [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "34612c2dd56544e48799380724ad75ef" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34612c2dd56544e48799380724ad75ef" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 37605 } } }
I20260923 15:02:28.143682 23383 sys_catalog.cc:458] T 00000000000000000000000000000000 P 34612c2dd56544e48799380724ad75ef [sys.catalog]: This master's current role is: LEADER
I20260923 15:02:28.143731 13614 external_mini_cluster.cc:1428] Setting key c367b05b6ee48966758f91f2bc40672c
I20260923 15:02:28.144028 23384 sys_catalog.cc:455] T 00000000000000000000000000000000 P 34612c2dd56544e48799380724ad75ef [sys.catalog]: SysCatalogTable state changed. Reason: New leader 34612c2dd56544e48799380724ad75ef. Latest consensus state: current_term: 1 leader_uuid: "34612c2dd56544e48799380724ad75ef" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "34612c2dd56544e48799380724ad75ef" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 37605 } } }
I20260923 15:02:28.144090 23384 sys_catalog.cc:458] T 00000000000000000000000000000000 P 34612c2dd56544e48799380724ad75ef [sys.catalog]: This master's current role is: LEADER
I20260923 15:02:28.144214 23391 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 15:02:28.145001 23391 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 15:02:28.146812 23400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:28.144376 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33086 (local address 127.13.75.190:37605)
0923 15:02:28.144583 (+ 207us) server_negotiation.cc:206] Beginning negotiation
0923 15:02:28.144593 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0923 15:02:28.144858 (+ 265us) server_negotiation.cc:407] Connection header received
0923 15:02:28.145029 (+ 171us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:02:28.145041 (+ 12us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:02:28.145093 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:02:28.145260 (+ 167us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:02:28.146133 (+ 873us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:02:28.146138 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:02:28.146138 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 15:02:28.146143 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:02:28.146192 (+ 49us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 15:02:28.146210 (+ 18us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:02:28.146285 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0923 15:02:28.146539 (+ 254us) server_negotiation.cc:299] Negotiation successful
0923 15:02:28.146607 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":114,"thread_start_us":62,"threads_started":1}
I20260923 15:02:28.146989 23391 catalog_manager.cc:1347] Generated new cluster ID: 88226d31b6704978b343eaeba7e85cd1
I20260923 15:02:28.147046 23391 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 15:02:28.152334 23391 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 15:02:28.152580 23391 catalog_manager.cc:1504] Loading token signing keys...
I20260923 15:02:28.159308 23391 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 34612c2dd56544e48799380724ad75ef: Generated new TSK 0
I20260923 15:02:28.159582 23391 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 15:02:28.168771 13614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKC4soy/build/release/bin/kudu
/tmp/dist-test-taskKC4soy/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790175691491353-13614-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790175691491353-13614-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790175691491353-13614-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790175691491353-13614-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.75.129:0
--local_ip_for_outbound_sockets=127.13.75.129
--webserver_interface=127.13.75.129
--webserver_port=0
--tserver_master_addrs=127.13.75.190:37605
--builtin_ntp_servers=127.13.75.148:37807
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260923 15:02:28.253896 23403 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:02:28.254135 23403 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:02:28.254163 23403 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:02:28.255530 23403 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:02:28.255590 23403 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:02:28.255620 23403 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.75.129
I20260923 15:02:28.257161 23403 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.75.148:37807
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790175691491353-13614-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790175691491353-13614-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.75.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790175691491353-13614-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.75.129
--webserver_port=0
--tserver_master_addrs=127.13.75.190:37605
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23403
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.75.129
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790175691491353-13614-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:02:28.257452 23403 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:02:28.257795 23403 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:02:28.264163 23408 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:02:28.264199 23411 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:02:28.264135 23409 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:02:28.264402 23403 server_base.cc:1034] running on GCE node
I20260923 15:02:28.264694 23403 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:02:28.264992 23403 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:02:28.266181 23403 hybrid_clock.cc:648] HybridClock initialized: now 1790175748266160 us; error 34 us; skew 500 ppm
I20260923 15:02:28.267642 23403 webserver.cc:460] Webserver started at http://127.13.75.129:42481/ using document root <none> and password file <none>
I20260923 15:02:28.267912 23403 fs_manager.cc:359] Metadata directory not provided
I20260923 15:02:28.267963 23403 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:02:28.268154 23403 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:02:28.269505 23403 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790175691491353-13614-0/minicluster-data/ts-0/data/instance:
uuid: "b1601ec2a01e4dc59fe69f3c3e1cb77c"
format_stamp: "Formatted at 2026-09-23 15:02:28 on dist-test-slave-cnb0"
server_key: "0cffdb5d822ef0d69e9a39c0b3f34119"
server_key_iv: "53cccd27ad65dfe8a36e51a3495d87c6"
server_key_version: "encryptionkey@0"
I20260923 15:02:28.269891 23403 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790175691491353-13614-0/minicluster-data/ts-0/wal/instance:
uuid: "b1601ec2a01e4dc59fe69f3c3e1cb77c"
format_stamp: "Formatted at 2026-09-23 15:02:28 on dist-test-slave-cnb0"
server_key: "0cffdb5d822ef0d69e9a39c0b3f34119"
server_key_iv: "53cccd27ad65dfe8a36e51a3495d87c6"
server_key_version: "encryptionkey@0"
I20260923 15:02:28.271397 23403 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.004s sys 0.000s
I20260923 15:02:28.272233 23417 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:02:28.272395 23403 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:02:28.272490 23403 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790175691491353-13614-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790175691491353-13614-0/minicluster-data/ts-0/wal
uuid: "b1601ec2a01e4dc59fe69f3c3e1cb77c"
format_stamp: "Formatted at 2026-09-23 15:02:28 on dist-test-slave-cnb0"
server_key: "0cffdb5d822ef0d69e9a39c0b3f34119"
server_key_iv: "53cccd27ad65dfe8a36e51a3495d87c6"
server_key_version: "encryptionkey@0"
I20260923 15:02:28.272583 23403 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790175691491353-13614-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790175691491353-13614-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790175691491353-13614-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:02:28.276165 23403 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:02:28.276475 23403 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:02:28.276830 23403 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:02:28.277144 23403 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 15:02:28.277561 23403 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 15:02:28.277616 23403 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 15:02:28.277659 23403 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 15:02:28.277688 23403 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 15:02:28.284102 23403 rpc_server.cc:307] RPC server started. Bound to: 127.13.75.129:40659
I20260923 15:02:28.284138 23530 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.75.129:40659 every 8 connection(s)
I20260923 15:02:28.284579 23403 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790175691491353-13614-0/minicluster-data/ts-0/data/info.pb
I20260923 15:02:28.288178 23400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:28.285387 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.129:55113 (local address 127.13.75.190:37605)
0923 15:02:28.285595 (+ 208us) server_negotiation.cc:206] Beginning negotiation
0923 15:02:28.285598 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:02:28.285811 (+ 213us) server_negotiation.cc:407] Connection header received
0923 15:02:28.286405 (+ 594us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:02:28.286407 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:02:28.286417 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:02:28.286473 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:02:28.287843 (+ 1370us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:02:28.287845 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:02:28.287845 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 15:02:28.287850 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:02:28.287868 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 15:02:28.287873 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:02:28.287944 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0923 15:02:28.288078 (+ 134us) server_negotiation.cc:299] Negotiation successful
0923 15:02:28.288099 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":171}
I20260923 15:02:28.288406 23533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:28.285289 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:37605 (local address 127.13.75.129:55113)
0923 15:02:28.285687 (+ 398us) negotiation.cc:107] Waiting for socket to connect
0923 15:02:28.285723 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0923 15:02:28.286259 (+ 536us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:02:28.286574 (+ 315us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:02:28.286578 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:02:28.286586 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:02:28.287660 (+ 1074us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 15:02:28.287660 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:02:28.287695 (+ 35us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 15:02:28.287727 (+ 32us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:02:28.287941 (+ 214us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:02:28.287948 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:02:28.287951 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 15:02:28.288057 (+ 106us) client_negotiation.cc:240] Negotiation successful
0923 15:02:28.288131 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":320,"thread_start_us":123,"threads_started":1}
I20260923 15:02:28.289136 23531 heartbeater.cc:347] Connected to a master server at 127.13.75.190:37605
I20260923 15:02:28.289240 23531 heartbeater.cc:464] Registering TS with master...
I20260923 15:02:28.289382 23531 heartbeater.cc:511] Master 127.13.75.190:37605 requested a full tablet report, sending...
I20260923 15:02:28.289963 23343 ts_manager.cc:194] Registered new tserver with Master: b1601ec2a01e4dc59fe69f3c3e1cb77c (127.13.75.129:40659)
I20260923 15:02:28.294761 13614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKC4soy/build/release/bin/kudu as pid 23403
I20260923 15:02:28.294947 13614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790175691491353-13614-0/minicluster-data/ts-0/wal/instance
I20260923 15:02:28.295153 13614 external_mini_cluster.cc:1428] Setting key 26d5f177a804dafcb4b013ea99d96b33
I20260923 15:02:28.296291 13614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKC4soy/build/release/bin/kudu
/tmp/dist-test-taskKC4soy/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790175691491353-13614-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790175691491353-13614-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790175691491353-13614-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790175691491353-13614-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.75.130:0
--local_ip_for_outbound_sockets=127.13.75.130
--webserver_interface=127.13.75.130
--webserver_port=0
--tserver_master_addrs=127.13.75.190:37605
--builtin_ntp_servers=127.13.75.148:37807
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
W20260923 15:02:28.378584 23534 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:02:28.378875 23534 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:02:28.378943 23534 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:02:28.380417 23534 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:02:28.380470 23534 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:02:28.380501 23534 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.75.130
I20260923 15:02:28.381934 23534 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.75.148:37807
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790175691491353-13614-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790175691491353-13614-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.75.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790175691491353-13614-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.75.130
--webserver_port=0
--tserver_master_addrs=127.13.75.190:37605
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23534
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.75.130
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790175691491353-13614-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:02:28.382166 23534 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:02:28.382439 23534 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:02:28.388096 23539 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:02:28.388127 23542 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:02:28.388096 23540 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:02:28.388402 23534 server_base.cc:1034] running on GCE node
I20260923 15:02:28.388686 23534 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:02:28.389029 23534 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:02:28.390192 23534 hybrid_clock.cc:648] HybridClock initialized: now 1790175748390169 us; error 40 us; skew 500 ppm
I20260923 15:02:28.391973 23534 webserver.cc:460] Webserver started at http://127.13.75.130:45125/ using document root <none> and password file <none>
I20260923 15:02:28.392277 23534 fs_manager.cc:359] Metadata directory not provided
I20260923 15:02:28.392355 23534 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:02:28.392522 23534 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:02:28.393767 23534 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790175691491353-13614-0/minicluster-data/ts-1/data/instance:
uuid: "944e5aba269242d4a4cfa9e7b352fb88"
format_stamp: "Formatted at 2026-09-23 15:02:28 on dist-test-slave-cnb0"
server_key: "c400a0b75bb037ec555c2f0ddf7bfcfe"
server_key_iv: "60a80fda47e95ed31ce8c7353d63a0b1"
server_key_version: "encryptionkey@0"
I20260923 15:02:28.394170 23534 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790175691491353-13614-0/minicluster-data/ts-1/wal/instance:
uuid: "944e5aba269242d4a4cfa9e7b352fb88"
format_stamp: "Formatted at 2026-09-23 15:02:28 on dist-test-slave-cnb0"
server_key: "c400a0b75bb037ec555c2f0ddf7bfcfe"
server_key_iv: "60a80fda47e95ed31ce8c7353d63a0b1"
server_key_version: "encryptionkey@0"
I20260923 15:02:28.395751 23534 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260923 15:02:28.396639 23548 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:02:28.396817 23534 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260923 15:02:28.396931 23534 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790175691491353-13614-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790175691491353-13614-0/minicluster-data/ts-1/wal
uuid: "944e5aba269242d4a4cfa9e7b352fb88"
format_stamp: "Formatted at 2026-09-23 15:02:28 on dist-test-slave-cnb0"
server_key: "c400a0b75bb037ec555c2f0ddf7bfcfe"
server_key_iv: "60a80fda47e95ed31ce8c7353d63a0b1"
server_key_version: "encryptionkey@0"
I20260923 15:02:28.397037 23534 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790175691491353-13614-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790175691491353-13614-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790175691491353-13614-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:02:28.400637 23534 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:02:28.400938 23534 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:02:28.401094 23534 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:02:28.401309 23534 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 15:02:28.401634 23534 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 15:02:28.401690 23534 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 15:02:28.401731 23534 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 15:02:28.401757 23534 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 15:02:28.407680 23534 rpc_server.cc:307] RPC server started. Bound to: 127.13.75.130:37317
I20260923 15:02:28.407773 23661 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.75.130:37317 every 8 connection(s)
I20260923 15:02:28.408116 23534 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790175691491353-13614-0/minicluster-data/ts-1/data/info.pb
I20260923 15:02:28.411257 13614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKC4soy/build/release/bin/kudu as pid 23534
I20260923 15:02:28.411386 13614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790175691491353-13614-0/minicluster-data/ts-1/wal/instance
I20260923 15:02:28.411571 23400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:28.409011 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.130:41817 (local address 127.13.75.190:37605)
0923 15:02:28.409108 (+ 97us) server_negotiation.cc:206] Beginning negotiation
0923 15:02:28.409111 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:02:28.409299 (+ 188us) server_negotiation.cc:407] Connection header received
0923 15:02:28.409878 (+ 579us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:02:28.409879 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:02:28.409885 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:02:28.409933 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:02:28.411294 (+ 1361us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:02:28.411296 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:02:28.411296 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 15:02:28.411302 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:02:28.411317 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 15:02:28.411321 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:02:28.411383 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0923 15:02:28.411493 (+ 110us) server_negotiation.cc:299] Negotiation successful
0923 15:02:28.411515 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":61}
I20260923 15:02:28.411608 13614 external_mini_cluster.cc:1428] Setting key ee2a8a9d719a1dc67f760527f551d6d4
I20260923 15:02:28.412007 23664 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:28.408932 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:37605 (local address 127.13.75.130:41817)
0923 15:02:28.409169 (+ 237us) negotiation.cc:107] Waiting for socket to connect
0923 15:02:28.409204 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0923 15:02:28.409763 (+ 559us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:02:28.410036 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:02:28.410040 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:02:28.410048 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:02:28.411131 (+ 1083us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 15:02:28.411132 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:02:28.411162 (+ 30us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 15:02:28.411193 (+ 31us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:02:28.411382 (+ 189us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:02:28.411387 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:02:28.411390 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 15:02:28.411700 (+ 310us) client_negotiation.cc:240] Negotiation successful
0923 15:02:28.411789 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":97,"threads_started":1}
I20260923 15:02:28.412474 23662 heartbeater.cc:347] Connected to a master server at 127.13.75.190:37605
I20260923 15:02:28.412576 23662 heartbeater.cc:464] Registering TS with master...
I20260923 15:02:28.412665 13614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKC4soy/build/release/bin/kudu
/tmp/dist-test-taskKC4soy/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790175691491353-13614-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790175691491353-13614-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790175691491353-13614-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790175691491353-13614-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.75.131:0
--local_ip_for_outbound_sockets=127.13.75.131
--webserver_interface=127.13.75.131
--webserver_port=0
--tserver_master_addrs=127.13.75.190:37605
--builtin_ntp_servers=127.13.75.148:37807
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_encryption=disabled
--rpc_authentication=disabled with env {}
I20260923 15:02:28.412742 23662 heartbeater.cc:511] Master 127.13.75.190:37605 requested a full tablet report, sending...
I20260923 15:02:28.413101 23343 ts_manager.cc:194] Registered new tserver with Master: 944e5aba269242d4a4cfa9e7b352fb88 (127.13.75.130:37317)
W20260923 15:02:28.497242 23665 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:02:28.497565 23665 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:02:28.497594 23665 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:02:28.498845 23665 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:02:28.498920 23665 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:02:28.498950 23665 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.75.131
I20260923 15:02:28.500469 23665 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.75.148:37807
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790175691491353-13614-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790175691491353-13614-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.75.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790175691491353-13614-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.75.131
--webserver_port=0
--tserver_master_addrs=127.13.75.190:37605
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23665
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.75.131
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790175691491353-13614-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:02:28.500757 23665 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:02:28.501135 23665 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:02:28.506583 23673 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:02:28.506594 23671 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:02:28.506609 23670 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:02:28.506853 23665 server_base.cc:1034] running on GCE node
I20260923 15:02:28.507189 23665 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:02:28.507491 23665 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:02:28.508651 23665 hybrid_clock.cc:648] HybridClock initialized: now 1790175748508620 us; error 49 us; skew 500 ppm
I20260923 15:02:28.510037 23665 webserver.cc:460] Webserver started at http://127.13.75.131:43111/ using document root <none> and password file <none>
I20260923 15:02:28.510288 23665 fs_manager.cc:359] Metadata directory not provided
I20260923 15:02:28.510334 23665 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:02:28.510488 23665 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:02:28.511749 23665 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790175691491353-13614-0/minicluster-data/ts-2/data/instance:
uuid: "fbbf4431aee34244915b8fbae161254c"
format_stamp: "Formatted at 2026-09-23 15:02:28 on dist-test-slave-cnb0"
server_key: "b9bf922c5b02bea63a05bdd5f264e1d5"
server_key_iv: "bf6631183fa93b4b9644459f01f633a2"
server_key_version: "encryptionkey@0"
I20260923 15:02:28.512148 23665 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790175691491353-13614-0/minicluster-data/ts-2/wal/instance:
uuid: "fbbf4431aee34244915b8fbae161254c"
format_stamp: "Formatted at 2026-09-23 15:02:28 on dist-test-slave-cnb0"
server_key: "b9bf922c5b02bea63a05bdd5f264e1d5"
server_key_iv: "bf6631183fa93b4b9644459f01f633a2"
server_key_version: "encryptionkey@0"
I20260923 15:02:28.513563 23665 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 15:02:28.514448 23679 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:02:28.514643 23665 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:02:28.514750 23665 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790175691491353-13614-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790175691491353-13614-0/minicluster-data/ts-2/wal
uuid: "fbbf4431aee34244915b8fbae161254c"
format_stamp: "Formatted at 2026-09-23 15:02:28 on dist-test-slave-cnb0"
server_key: "b9bf922c5b02bea63a05bdd5f264e1d5"
server_key_iv: "bf6631183fa93b4b9644459f01f633a2"
server_key_version: "encryptionkey@0"
I20260923 15:02:28.514808 23665 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790175691491353-13614-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790175691491353-13614-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790175691491353-13614-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:02:28.518585 23665 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:02:28.518826 23665 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:02:28.519011 23665 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:02:28.519198 23665 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 15:02:28.519474 23665 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 15:02:28.519505 23665 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 15:02:28.519526 23665 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 15:02:28.519539 23665 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 15:02:28.525178 23665 rpc_server.cc:307] RPC server started. Bound to: 127.13.75.131:44355
I20260923 15:02:28.525269 23792 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.75.131:44355 every 8 connection(s)
I20260923 15:02:28.525627 23665 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790175691491353-13614-0/minicluster-data/ts-2/data/info.pb
I20260923 15:02:28.528071 13614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKC4soy/build/release/bin/kudu as pid 23665
I20260923 15:02:28.528179 13614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireEncryptionInsecureCluster.1790175691491353-13614-0/minicluster-data/ts-2/wal/instance
I20260923 15:02:28.528385 13614 external_mini_cluster.cc:1428] Setting key 9395b8067128948c102f97ffd84ecbff
I20260923 15:02:28.529675 23400 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:28.526641 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.131:60187 (local address 127.13.75.190:37605)
0923 15:02:28.526718 (+ 77us) server_negotiation.cc:206] Beginning negotiation
0923 15:02:28.526721 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:02:28.527066 (+ 345us) server_negotiation.cc:407] Connection header received
0923 15:02:28.527774 (+ 708us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:02:28.527776 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:02:28.527785 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:02:28.527865 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:02:28.529411 (+ 1546us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:02:28.529413 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:02:28.529413 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 15:02:28.529418 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:02:28.529435 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 15:02:28.529440 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:02:28.529496 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0923 15:02:28.529580 (+ 84us) server_negotiation.cc:299] Negotiation successful
0923 15:02:28.529598 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260923 15:02:28.529903 23795 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:28.526593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:37605 (local address 127.13.75.131:60187)
0923 15:02:28.526863 (+ 270us) negotiation.cc:107] Waiting for socket to connect
0923 15:02:28.526957 (+ 94us) client_negotiation.cc:174] Beginning negotiation
0923 15:02:28.527625 (+ 668us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:02:28.527949 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:02:28.527954 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:02:28.527973 (+ 19us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:02:28.529235 (+ 1262us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 15:02:28.529235 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:02:28.529284 (+ 49us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 15:02:28.529316 (+ 32us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:02:28.529493 (+ 177us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:02:28.529498 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:02:28.529502 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 15:02:28.529579 (+ 77us) client_negotiation.cc:240] Negotiation successful
0923 15:02:28.529650 (+ 71us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":194,"thread_start_us":117,"threads_started":1}
I20260923 15:02:28.530431 23793 heartbeater.cc:347] Connected to a master server at 127.13.75.190:37605
I20260923 15:02:28.530552 23793 heartbeater.cc:464] Registering TS with master...
I20260923 15:02:28.530822 23793 heartbeater.cc:511] Master 127.13.75.190:37605 requested a full tablet report, sending...
I20260923 15:02:28.531253 23343 ts_manager.cc:194] Registered new tserver with Master: fbbf4431aee34244915b8fbae161254c (127.13.75.131:44355)
I20260923 15:02:28.540879 13614 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260923 15:02:28.543186 13614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKC4soy/build/release/bin/kudu with pid 23403
W20260923 15:02:28.543179 23400 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 15:02:28.542425 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33098 (local address 127.13.75.190:37605)
0923 15:02:28.542500 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0923 15:02:28.542503 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:02:28.542707 (+ 204us) server_negotiation.cc:407] Connection header received
0923 15:02:28.542740 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:02:28.542740 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:02:28.542746 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:02:28.542808 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:02:28.543087 (+ 279us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:33098: BlockingRecv error: recv got EOF from 127.0.0.1:33098 (error 108)
Metrics: {"server-negotiator.queue_time_us":44}
I20260923 15:02:28.548477 13614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKC4soy/build/release/bin/kudu with pid 23534
I20260923 15:02:28.553176 13614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKC4soy/build/release/bin/kudu with pid 23665
I20260923 15:02:28.557752 13614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKC4soy/build/release/bin/kudu with pid 23312
2026-09-23T15:02:28Z chronyd exiting
[ OK ] SecurityITest.TestRequireEncryptionInsecureCluster (568 ms)
[ RUN ] SecurityITest.TestRequireAuthenticationSecureCluster
Loading random data
Initializing database '/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175691491353-13614-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 15:02:28 dist-test-slave-cnb0 krb5kdc[23803](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 15:02:28 dist-test-slave-cnb0 krb5kdc[23803](info): set up 2 sockets
Sep 23 15:02:28 dist-test-slave-cnb0 krb5kdc[23803](info): commencing operation
krb5kdc: starting...
W20260923 15:02:29.617060 13614 mini_kdc.cc:121] Time spent starting KDC: real 1.039s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175691491353-13614-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175691491353-13614-0/krb5kdc/test-user.keytab.
Sep 23 15:02:29 dist-test-slave-cnb0 krb5kdc[23803](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175749, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T15:02:29Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T15:02:29Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.75.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.75.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.75.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.190.keytab.
Entry for principal kudu/127.13.75.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.190.keytab.
I20260923 15:02:29.751792 13614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKC4soy/build/release/bin/kudu
/tmp/dist-test-taskKC4soy/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175691491353-13614-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175691491353-13614-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175691491353-13614-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175691491353-13614-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.75.190:38207
--webserver_interface=127.13.75.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.190.keytab
--principal=kudu/127.13.75.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.75.148:39165
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.75.190:38207
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175691491353-13614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175691491353-13614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175691491353-13614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:02:29.837545 23816 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:02:29.837806 23816 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:02:29.837858 23816 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:02:29.839227 23816 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 15:02:29.839298 23816 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:02:29.839322 23816 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:02:29.839344 23816 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 15:02:29.839371 23816 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 15:02:29.841079 23816 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.75.148:39165
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175691491353-13614-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175691491353-13614-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.75.190:38207
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.190.keytab
--principal=kudu/127.13.75.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.75.190:38207
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175691491353-13614-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.75.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23816
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175691491353-13614-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:02:29.841406 23816 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:02:29.841672 23816 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:02:29.846745 23821 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:02:29.846720 23822 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:02:29.847038 23824 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:02:29.847078 23816 server_base.cc:1034] running on GCE node
I20260923 15:02:29.847393 23816 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:02:29.847751 23816 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:02:29.848929 23816 hybrid_clock.cc:648] HybridClock initialized: now 1790175749848922 us; error 41 us; skew 500 ppm
Sep 23 15:02:29 dist-test-slave-cnb0 krb5kdc[23803](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175749, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:02:29.851744 23816 init.cc:377] Logged in from keytab as kudu/127.13.75.190@KRBTEST.COM (short username kudu)
I20260923 15:02:29.852295 23816 webserver.cc:460] Webserver started at http://127.13.75.190:43207/ using document root <none> and password file <none>
I20260923 15:02:29.852540 23816 fs_manager.cc:359] Metadata directory not provided
I20260923 15:02:29.852613 23816 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:02:29.852773 23816 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:02:29.853924 23816 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175691491353-13614-0/minicluster-data/master-0/data/instance:
uuid: "15fcd72280b64a03b72ad34341772115"
format_stamp: "Formatted at 2026-09-23 15:02:29 on dist-test-slave-cnb0"
server_key: "0072121fa45971af4c682a6c63e37e88"
server_key_iv: "91a45fedbb022553643d95f28b7906d5"
server_key_version: "encryptionkey@0"
I20260923 15:02:29.854346 23816 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175691491353-13614-0/minicluster-data/master-0/wal/instance:
uuid: "15fcd72280b64a03b72ad34341772115"
format_stamp: "Formatted at 2026-09-23 15:02:29 on dist-test-slave-cnb0"
server_key: "0072121fa45971af4c682a6c63e37e88"
server_key_iv: "91a45fedbb022553643d95f28b7906d5"
server_key_version: "encryptionkey@0"
I20260923 15:02:29.855886 23816 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260923 15:02:29.856820 23831 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:02:29.857028 23816 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260923 15:02:29.857160 23816 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175691491353-13614-0/minicluster-data/master-0/data,/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175691491353-13614-0/minicluster-data/master-0/wal
uuid: "15fcd72280b64a03b72ad34341772115"
format_stamp: "Formatted at 2026-09-23 15:02:29 on dist-test-slave-cnb0"
server_key: "0072121fa45971af4c682a6c63e37e88"
server_key_iv: "91a45fedbb022553643d95f28b7906d5"
server_key_version: "encryptionkey@0"
I20260923 15:02:29.857245 23816 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175691491353-13614-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175691491353-13614-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175691491353-13614-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:02:29.879551 23816 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:02:29.881994 23816 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:02:29.882179 23816 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:02:29.886466 23816 rpc_server.cc:307] RPC server started. Bound to: 127.13.75.190:38207
I20260923 15:02:29.886510 23883 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.75.190:38207 every 8 connection(s)
I20260923 15:02:29.886931 23816 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175691491353-13614-0/minicluster-data/master-0/data/info.pb
I20260923 15:02:29.887321 13614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKC4soy/build/release/bin/kudu as pid 23816
I20260923 15:02:29.887404 13614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175691491353-13614-0/minicluster-data/master-0/wal/instance
I20260923 15:02:29.887625 13614 external_mini_cluster.cc:1428] Setting key 2a5838358e735b856642004649c954a2
I20260923 15:02:29.888309 23884 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:02:29.891729 23884 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 15fcd72280b64a03b72ad34341772115: Bootstrap starting.
I20260923 15:02:29.892416 23884 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 15fcd72280b64a03b72ad34341772115: Neither blocks nor log segments found. Creating new log.
I20260923 15:02:29.892738 23884 log.cc:824] T 00000000000000000000000000000000 P 15fcd72280b64a03b72ad34341772115: Log is configured to *not* fsync() on all Append() calls
I20260923 15:02:29.893553 23884 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 15fcd72280b64a03b72ad34341772115: No bootstrap required, opened a new log
Sep 23 15:02:29 dist-test-slave-cnb0 krb5kdc[23803](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175749, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.75.190@KRBTEST.COM
I20260923 15:02:29.895849 23884 raft_consensus.cc:348] T 00000000000000000000000000000000 P 15fcd72280b64a03b72ad34341772115 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15fcd72280b64a03b72ad34341772115" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 38207 } }
I20260923 15:02:29.896066 23884 raft_consensus.cc:374] T 00000000000000000000000000000000 P 15fcd72280b64a03b72ad34341772115 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:02:29.896122 23884 raft_consensus.cc:729] T 00000000000000000000000000000000 P 15fcd72280b64a03b72ad34341772115 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 15fcd72280b64a03b72ad34341772115, State: Initialized, Role: FOLLOWER
I20260923 15:02:29.896281 23884 consensus_queue.cc:260] T 00000000000000000000000000000000 P 15fcd72280b64a03b72ad34341772115 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15fcd72280b64a03b72ad34341772115" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 38207 } }
I20260923 15:02:29.896387 23884 raft_consensus.cc:388] T 00000000000000000000000000000000 P 15fcd72280b64a03b72ad34341772115 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 15:02:29.896517 23884 raft_consensus.cc:482] T 00000000000000000000000000000000 P 15fcd72280b64a03b72ad34341772115 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 15:02:29.896590 23884 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 15fcd72280b64a03b72ad34341772115 [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:02:29.897255 23884 raft_consensus.cc:504] T 00000000000000000000000000000000 P 15fcd72280b64a03b72ad34341772115 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15fcd72280b64a03b72ad34341772115" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 38207 } }
I20260923 15:02:29.897349 23884 leader_election.cc:302] T 00000000000000000000000000000000 P 15fcd72280b64a03b72ad34341772115 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 15fcd72280b64a03b72ad34341772115; no voters:
I20260923 15:02:29.897573 23884 leader_election.cc:288] T 00000000000000000000000000000000 P 15fcd72280b64a03b72ad34341772115 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 15:02:29.897641 23889 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 15fcd72280b64a03b72ad34341772115 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 15:02:29.897822 23889 raft_consensus.cc:686] T 00000000000000000000000000000000 P 15fcd72280b64a03b72ad34341772115 [term 1 LEADER]: Becoming Leader. State: Replica: 15fcd72280b64a03b72ad34341772115, State: Running, Role: LEADER
I20260923 15:02:29.897890 23884 sys_catalog.cc:564] T 00000000000000000000000000000000 P 15fcd72280b64a03b72ad34341772115 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 15:02:29.897957 23889 consensus_queue.cc:237] T 00000000000000000000000000000000 P 15fcd72280b64a03b72ad34341772115 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15fcd72280b64a03b72ad34341772115" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 38207 } }
I20260923 15:02:29.898348 23891 sys_catalog.cc:455] T 00000000000000000000000000000000 P 15fcd72280b64a03b72ad34341772115 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 15fcd72280b64a03b72ad34341772115. Latest consensus state: current_term: 1 leader_uuid: "15fcd72280b64a03b72ad34341772115" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15fcd72280b64a03b72ad34341772115" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 38207 } } }
I20260923 15:02:29.898454 23891 sys_catalog.cc:458] T 00000000000000000000000000000000 P 15fcd72280b64a03b72ad34341772115 [sys.catalog]: This master's current role is: LEADER
I20260923 15:02:29.898684 23896 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 15:02:29.899312 23896 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 15:02:29.899588 23887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:29.888245 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58752 (local address 127.13.75.190:38207)
0923 15:02:29.888460 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0923 15:02:29.888470 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0923 15:02:29.888505 (+ 35us) server_negotiation.cc:407] Connection header received
0923 15:02:29.888956 (+ 451us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:02:29.888962 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:02:29.889049 (+ 87us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:02:29.889172 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:02:29.890017 (+ 845us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:29.891014 (+ 997us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:29.892100 (+ 1086us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:29.892584 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:29.892715 (+ 131us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:29.895208 (+ 2493us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:02:29.895213 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:02:29.895213 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:02:29.895226 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:02:29.897564 (+ 2338us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:02:29.898063 (+ 499us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:02:29.898066 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:02:29.898066 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:02:29.898174 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:02:29.898439 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:02:29.898440 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:02:29.898440 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:02:29.898816 (+ 376us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:02:29.899057 (+ 241us) server_negotiation.cc:1035] Waiting for connection context
0923 15:02:29.899158 (+ 101us) server_negotiation.cc:299] Negotiation successful
0923 15:02:29.899338 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":87,"threads_started":1}
I20260923 15:02:29.899854 23890 sys_catalog.cc:455] T 00000000000000000000000000000000 P 15fcd72280b64a03b72ad34341772115 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "15fcd72280b64a03b72ad34341772115" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "15fcd72280b64a03b72ad34341772115" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 38207 } } }
I20260923 15:02:29.899940 23890 sys_catalog.cc:458] T 00000000000000000000000000000000 P 15fcd72280b64a03b72ad34341772115 [sys.catalog]: This master's current role is: LEADER
I20260923 15:02:29.903920 23906 mvcc.cc:205] Tried to move back new op lower bound from 7332559871592816640 to 7332559871582699520. Current Snapshot: MvccSnapshot[applied={T|T < 7332559871592816640}]
I20260923 15:02:29.904382 23896 catalog_manager.cc:1347] Generated new cluster ID: 308465286d6843ada3315b564fe53d3e
I20260923 15:02:29.904465 23896 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 15:02:29.910580 23896 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 15:02:29.911221 23896 catalog_manager.cc:1504] Loading token signing keys...
I20260923 15:02:29.913707 23896 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 15fcd72280b64a03b72ad34341772115: Generated new TSK 0
I20260923 15:02:29.913918 23896 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.75.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.75.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.75.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.129.keytab.
Entry for principal kudu/127.13.75.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.129.keytab.
I20260923 15:02:29.943624 13614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKC4soy/build/release/bin/kudu
/tmp/dist-test-taskKC4soy/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175691491353-13614-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175691491353-13614-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175691491353-13614-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175691491353-13614-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.75.129:0
--local_ip_for_outbound_sockets=127.13.75.129
--webserver_interface=127.13.75.129
--webserver_port=0
--tserver_master_addrs=127.13.75.190:38207
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.129.keytab
--principal=kudu/127.13.75.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.75.148:39165
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175691491353-13614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175691491353-13614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175691491353-13614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:02:30.033926 23910 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:02:30.034193 23910 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:02:30.034240 23910 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:02:30.035611 23910 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:02:30.035686 23910 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:02:30.035734 23910 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.75.129
I20260923 15:02:30.037436 23910 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.75.148:39165
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175691491353-13614-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175691491353-13614-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.129.keytab
--principal=kudu/127.13.75.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.75.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175691491353-13614-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.75.129
--webserver_port=0
--tserver_master_addrs=127.13.75.190:38207
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.23910
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.75.129
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175691491353-13614-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:02:30.037762 23910 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:02:30.038064 23910 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:02:30.044672 23918 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:02:30.044860 23916 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:02:30.045079 23915 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:02:30.045469 23910 server_base.cc:1034] running on GCE node
I20260923 15:02:30.045701 23910 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:02:30.045998 23910 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:02:30.047214 23910 hybrid_clock.cc:648] HybridClock initialized: now 1790175750047186 us; error 34 us; skew 500 ppm
Sep 23 15:02:30 dist-test-slave-cnb0 krb5kdc[23803](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175750, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:02:30.050628 23910 init.cc:377] Logged in from keytab as kudu/127.13.75.129@KRBTEST.COM (short username kudu)
I20260923 15:02:30.051230 23910 webserver.cc:460] Webserver started at http://127.13.75.129:39539/ using document root <none> and password file <none>
I20260923 15:02:30.051507 23910 fs_manager.cc:359] Metadata directory not provided
I20260923 15:02:30.051581 23910 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:02:30.051738 23910 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:02:30.052994 23910 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175691491353-13614-0/minicluster-data/ts-0/data/instance:
uuid: "816ad4b2ce35454f892ed20c7bed3a57"
format_stamp: "Formatted at 2026-09-23 15:02:30 on dist-test-slave-cnb0"
server_key: "79e2c6b69ff21c2e69dc05213a2cb1a6"
server_key_iv: "2aac34597751d9d0062ab800af2362f0"
server_key_version: "encryptionkey@0"
I20260923 15:02:30.053369 23910 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175691491353-13614-0/minicluster-data/ts-0/wal/instance:
uuid: "816ad4b2ce35454f892ed20c7bed3a57"
format_stamp: "Formatted at 2026-09-23 15:02:30 on dist-test-slave-cnb0"
server_key: "79e2c6b69ff21c2e69dc05213a2cb1a6"
server_key_iv: "2aac34597751d9d0062ab800af2362f0"
server_key_version: "encryptionkey@0"
I20260923 15:02:30.054872 23910 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 15:02:30.056000 23925 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:02:30.056286 23910 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260923 15:02:30.056412 23910 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175691491353-13614-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175691491353-13614-0/minicluster-data/ts-0/wal
uuid: "816ad4b2ce35454f892ed20c7bed3a57"
format_stamp: "Formatted at 2026-09-23 15:02:30 on dist-test-slave-cnb0"
server_key: "79e2c6b69ff21c2e69dc05213a2cb1a6"
server_key_iv: "2aac34597751d9d0062ab800af2362f0"
server_key_version: "encryptionkey@0"
I20260923 15:02:30.056530 23910 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175691491353-13614-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175691491353-13614-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175691491353-13614-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:02:30.064193 23910 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:02:30.067329 23910 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:02:30.067625 23910 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:02:30.067938 23910 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 15:02:30.068316 23910 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 15:02:30.068375 23910 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 15:02:30.068421 23910 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 15:02:30.068452 23910 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 15:02:30.077170 23910 rpc_server.cc:307] RPC server started. Bound to: 127.13.75.129:35885
I20260923 15:02:30.077210 24039 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.75.129:35885 every 8 connection(s)
I20260923 15:02:30.077607 23910 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175691491353-13614-0/minicluster-data/ts-0/data/info.pb
I20260923 15:02:30.079892 13614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKC4soy/build/release/bin/kudu as pid 23910
I20260923 15:02:30.079981 13614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175691491353-13614-0/minicluster-data/ts-0/wal/instance
I20260923 15:02:30.080195 13614 external_mini_cluster.cc:1428] Setting key 53c8ec9cb5d8360443f62f0b10069b8c
Sep 23 15:02:30 dist-test-slave-cnb0 krb5kdc[23803](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175750, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.129@KRBTEST.COM for kudu/127.13.75.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.13.75.130@KRBTEST.COM; defaulting to no policy
I20260923 15:02:30.090745 23887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:30.078415 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.129:35229 (local address 127.13.75.190:38207)
0923 15:02:30.078490 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0923 15:02:30.078492 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:02:30.078743 (+ 251us) server_negotiation.cc:407] Connection header received
0923 15:02:30.079424 (+ 681us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:02:30.079426 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:02:30.079438 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:02:30.079547 (+ 109us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:02:30.080837 (+ 1290us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:30.081410 (+ 573us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:30.082869 (+ 1459us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:30.083239 (+ 370us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:30.083314 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:30.086336 (+ 3022us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:02:30.086338 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:02:30.086338 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:02:30.086347 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:02:30.088883 (+ 2536us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:02:30.089363 (+ 480us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:02:30.089364 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:02:30.089365 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:02:30.089435 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:02:30.089862 (+ 427us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:02:30.089864 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:02:30.089864 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:02:30.090032 (+ 168us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:02:30.090122 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0923 15:02:30.090521 (+ 399us) server_negotiation.cc:299] Negotiation successful
0923 15:02:30.090644 (+ 123us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260923 15:02:30.090987 24042 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:30.078390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:38207 (local address 127.13.75.129:35229)
0923 15:02:30.078629 (+ 239us) negotiation.cc:107] Waiting for socket to connect
0923 15:02:30.078670 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0923 15:02:30.079264 (+ 594us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:02:30.079669 (+ 405us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:02:30.079674 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:02:30.080496 (+ 822us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:02:30.080734 (+ 238us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:30.080739 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:30.081523 (+ 784us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:30.081525 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:30.082749 (+ 1224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:30.082753 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:30.083342 (+ 589us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:30.083343 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:30.083515 (+ 172us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:30.084154 (+ 639us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:02:30.084166 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:02:30.086180 (+ 2014us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:02:30.089001 (+ 2821us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:02:30.089014 (+ 13us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:02:30.089017 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:02:30.089275 (+ 258us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:02:30.089583 (+ 308us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:02:30.089584 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:02:30.089584 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:02:30.089776 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:02:30.090113 (+ 337us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:02:30.090123 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:02:30.090287 (+ 164us) client_negotiation.cc:769] Sending connection context
0923 15:02:30.090486 (+ 199us) client_negotiation.cc:240] Negotiation successful
0923 15:02:30.090679 (+ 193us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":196,"thread_start_us":137,"threads_started":1}
I20260923 15:02:30.091693 24040 heartbeater.cc:347] Connected to a master server at 127.13.75.190:38207
I20260923 15:02:30.091842 24040 heartbeater.cc:464] Registering TS with master...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.75.130@KRBTEST.COM" created.
I20260923 15:02:30.092166 24040 heartbeater.cc:511] Master 127.13.75.190:38207 requested a full tablet report, sending...
I20260923 15:02:30.092923 23848 ts_manager.cc:194] Registered new tserver with Master: 816ad4b2ce35454f892ed20c7bed3a57 (127.13.75.129:35885)
I20260923 15:02:30.093886 23848 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.75.129@KRBTEST.COM'} at 127.13.75.129:35229
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.75.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.130.keytab.
Entry for principal kudu/127.13.75.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.130.keytab.
I20260923 15:02:30.115041 13614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKC4soy/build/release/bin/kudu
/tmp/dist-test-taskKC4soy/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175691491353-13614-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175691491353-13614-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175691491353-13614-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175691491353-13614-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.75.130:0
--local_ip_for_outbound_sockets=127.13.75.130
--webserver_interface=127.13.75.130
--webserver_port=0
--tserver_master_addrs=127.13.75.190:38207
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.130.keytab
--principal=kudu/127.13.75.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.75.148:39165
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175691491353-13614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175691491353-13614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175691491353-13614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:02:30.206975 24045 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:02:30.207207 24045 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:02:30.207234 24045 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:02:30.208691 24045 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:02:30.208773 24045 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:02:30.208807 24045 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.75.130
I20260923 15:02:30.210405 24045 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.75.148:39165
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175691491353-13614-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175691491353-13614-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.130.keytab
--principal=kudu/127.13.75.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.75.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175691491353-13614-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.75.130
--webserver_port=0
--tserver_master_addrs=127.13.75.190:38207
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24045
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.75.130
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175691491353-13614-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:02:30.210714 24045 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:02:30.210954 24045 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260923 15:02:30.217131 24045 server_base.cc:1034] running on GCE node
W20260923 15:02:30.217094 24050 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:02:30.217094 24051 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:02:30.217123 24053 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:02:30.217624 24045 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:02:30.217929 24045 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:02:30.219101 24045 hybrid_clock.cc:648] HybridClock initialized: now 1790175750219099 us; error 47 us; skew 500 ppm
Sep 23 15:02:30 dist-test-slave-cnb0 krb5kdc[23803](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175750, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:02:30.222119 24045 init.cc:377] Logged in from keytab as kudu/127.13.75.130@KRBTEST.COM (short username kudu)
I20260923 15:02:30.222671 24045 webserver.cc:460] Webserver started at http://127.13.75.130:43649/ using document root <none> and password file <none>
I20260923 15:02:30.222980 24045 fs_manager.cc:359] Metadata directory not provided
I20260923 15:02:30.223054 24045 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:02:30.223199 24045 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:02:30.224326 24045 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175691491353-13614-0/minicluster-data/ts-1/data/instance:
uuid: "23e192b3e46e4ed690b9b022e5594f44"
format_stamp: "Formatted at 2026-09-23 15:02:30 on dist-test-slave-cnb0"
server_key: "6f1fb9b3475ada95580c5c8efd19b963"
server_key_iv: "6ea4ee06fc32be5f5507c3fd220d5743"
server_key_version: "encryptionkey@0"
I20260923 15:02:30.224711 24045 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175691491353-13614-0/minicluster-data/ts-1/wal/instance:
uuid: "23e192b3e46e4ed690b9b022e5594f44"
format_stamp: "Formatted at 2026-09-23 15:02:30 on dist-test-slave-cnb0"
server_key: "6f1fb9b3475ada95580c5c8efd19b963"
server_key_iv: "6ea4ee06fc32be5f5507c3fd220d5743"
server_key_version: "encryptionkey@0"
I20260923 15:02:30.226487 24045 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.003s sys 0.000s
I20260923 15:02:30.227625 24060 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:02:30.227873 24045 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:02:30.228034 24045 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175691491353-13614-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175691491353-13614-0/minicluster-data/ts-1/wal
uuid: "23e192b3e46e4ed690b9b022e5594f44"
format_stamp: "Formatted at 2026-09-23 15:02:30 on dist-test-slave-cnb0"
server_key: "6f1fb9b3475ada95580c5c8efd19b963"
server_key_iv: "6ea4ee06fc32be5f5507c3fd220d5743"
server_key_version: "encryptionkey@0"
I20260923 15:02:30.228127 24045 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175691491353-13614-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175691491353-13614-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175691491353-13614-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:02:30.243798 24045 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:02:30.246225 24045 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:02:30.246414 24045 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:02:30.246730 24045 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 15:02:30.247125 24045 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 15:02:30.247184 24045 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 15:02:30.247231 24045 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 15:02:30.247277 24045 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 15:02:30.253717 24045 rpc_server.cc:307] RPC server started. Bound to: 127.13.75.130:45175
I20260923 15:02:30.253758 24173 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.75.130:45175 every 8 connection(s)
I20260923 15:02:30.254103 24045 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175691491353-13614-0/minicluster-data/ts-1/data/info.pb
Sep 23 15:02:30 dist-test-slave-cnb0 krb5kdc[23803](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175750, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.130@KRBTEST.COM for kudu/127.13.75.190@KRBTEST.COM
I20260923 15:02:30.261791 13614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKC4soy/build/release/bin/kudu as pid 24045
I20260923 15:02:30.261904 13614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175691491353-13614-0/minicluster-data/ts-1/wal/instance
I20260923 15:02:30.262156 13614 external_mini_cluster.cc:1428] Setting key 453593996d70f0bf722676a4d7339349
I20260923 15:02:30.265254 23887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:30.254853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.130:33121 (local address 127.13.75.190:38207)
0923 15:02:30.254929 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0923 15:02:30.254931 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:02:30.255209 (+ 278us) server_negotiation.cc:407] Connection header received
0923 15:02:30.255761 (+ 552us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:02:30.255763 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:02:30.255775 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:02:30.255843 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:02:30.256972 (+ 1129us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:30.257526 (+ 554us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:30.258650 (+ 1124us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:30.258853 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:30.259008 (+ 155us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:30.261474 (+ 2466us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:02:30.261475 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:02:30.261476 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:02:30.261485 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:02:30.263613 (+ 2128us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:02:30.264144 (+ 531us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:02:30.264145 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:02:30.264146 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:02:30.264217 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:02:30.264544 (+ 327us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:02:30.264545 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:02:30.264545 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:02:30.264665 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:02:30.264745 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0923 15:02:30.265053 (+ 308us) server_negotiation.cc:299] Negotiation successful
0923 15:02:30.265156 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260923 15:02:30.265472 24176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:30.254818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:38207 (local address 127.13.75.130:33121)
0923 15:02:30.255043 (+ 225us) negotiation.cc:107] Waiting for socket to connect
0923 15:02:30.255103 (+ 60us) client_negotiation.cc:174] Beginning negotiation
0923 15:02:30.255638 (+ 535us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:02:30.255967 (+ 329us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:02:30.255972 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:02:30.256635 (+ 663us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:02:30.256855 (+ 220us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:30.256861 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:30.257648 (+ 787us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:30.257649 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:30.258531 (+ 882us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:30.258533 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:30.259041 (+ 508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:30.259042 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:30.259158 (+ 116us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:30.259740 (+ 582us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:02:30.259749 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:02:30.261355 (+ 1606us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:02:30.263731 (+ 2376us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:02:30.263746 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:02:30.263749 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:02:30.264047 (+ 298us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:02:30.264290 (+ 243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:02:30.264291 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:02:30.264291 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:02:30.264420 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:02:30.264745 (+ 325us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:02:30.264754 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:02:30.264904 (+ 150us) client_negotiation.cc:769] Sending connection context
0923 15:02:30.265030 (+ 126us) client_negotiation.cc:240] Negotiation successful
0923 15:02:30.265209 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":187,"thread_start_us":132,"threads_started":1}
I20260923 15:02:30.266299 24174 heartbeater.cc:347] Connected to a master server at 127.13.75.190:38207
I20260923 15:02:30.266463 24174 heartbeater.cc:464] Registering TS with master...
I20260923 15:02:30.266742 24174 heartbeater.cc:511] Master 127.13.75.190:38207 requested a full tablet report, sending...
I20260923 15:02:30.267285 23848 ts_manager.cc:194] Registered new tserver with Master: 23e192b3e46e4ed690b9b022e5594f44 (127.13.75.130:45175)
I20260923 15:02:30.267716 23848 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.75.130@KRBTEST.COM'} at 127.13.75.130:33121
WARNING: no policy specified for kudu/127.13.75.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.75.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.75.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.131.keytab.
Entry for principal kudu/127.13.75.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.131.keytab.
I20260923 15:02:30.294775 13614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKC4soy/build/release/bin/kudu
/tmp/dist-test-taskKC4soy/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175691491353-13614-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175691491353-13614-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175691491353-13614-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175691491353-13614-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.75.131:0
--local_ip_for_outbound_sockets=127.13.75.131
--webserver_interface=127.13.75.131
--webserver_port=0
--tserver_master_addrs=127.13.75.190:38207
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.131.keytab
--principal=kudu/127.13.75.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.75.148:39165
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175691491353-13614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175691491353-13614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175691491353-13614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:02:30.378947 24179 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:02:30.379166 24179 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:02:30.379194 24179 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:02:30.380483 24179 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:02:30.380538 24179 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:02:30.380568 24179 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.75.131
I20260923 15:02:30.382043 24179 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.75.148:39165
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175691491353-13614-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175691491353-13614-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.131.keytab
--principal=kudu/127.13.75.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.75.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175691491353-13614-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.75.131
--webserver_port=0
--tserver_master_addrs=127.13.75.190:38207
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24179
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.75.131
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175691491353-13614-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:02:30.382252 24179 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:02:30.382447 24179 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:02:30.387794 24184 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:02:30.387905 24179 server_base.cc:1034] running on GCE node
W20260923 15:02:30.387800 24185 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:02:30.387816 24187 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:02:30.388288 24179 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:02:30.388615 24179 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:02:30.389825 24179 hybrid_clock.cc:648] HybridClock initialized: now 1790175750389773 us; error 80 us; skew 500 ppm
Sep 23 15:02:30 dist-test-slave-cnb0 krb5kdc[23803](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175750, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:02:30.392946 24179 init.cc:377] Logged in from keytab as kudu/127.13.75.131@KRBTEST.COM (short username kudu)
I20260923 15:02:30.393429 24179 webserver.cc:460] Webserver started at http://127.13.75.131:34841/ using document root <none> and password file <none>
I20260923 15:02:30.393697 24179 fs_manager.cc:359] Metadata directory not provided
I20260923 15:02:30.393745 24179 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:02:30.393882 24179 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:02:30.395145 24179 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175691491353-13614-0/minicluster-data/ts-2/data/instance:
uuid: "3e66c62e030e43de816586cf5aee513c"
format_stamp: "Formatted at 2026-09-23 15:02:30 on dist-test-slave-cnb0"
server_key: "0ee2571836f2117a9bdec6d10915bdd5"
server_key_iv: "a2814dadcaa9915193a25cc9e5d03d53"
server_key_version: "encryptionkey@0"
I20260923 15:02:30.395511 24179 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175691491353-13614-0/minicluster-data/ts-2/wal/instance:
uuid: "3e66c62e030e43de816586cf5aee513c"
format_stamp: "Formatted at 2026-09-23 15:02:30 on dist-test-slave-cnb0"
server_key: "0ee2571836f2117a9bdec6d10915bdd5"
server_key_iv: "a2814dadcaa9915193a25cc9e5d03d53"
server_key_version: "encryptionkey@0"
I20260923 15:02:30.396909 24179 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:02:30.397818 24194 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:02:30.398149 24179 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:02:30.398272 24179 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175691491353-13614-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175691491353-13614-0/minicluster-data/ts-2/wal
uuid: "3e66c62e030e43de816586cf5aee513c"
format_stamp: "Formatted at 2026-09-23 15:02:30 on dist-test-slave-cnb0"
server_key: "0ee2571836f2117a9bdec6d10915bdd5"
server_key_iv: "a2814dadcaa9915193a25cc9e5d03d53"
server_key_version: "encryptionkey@0"
I20260923 15:02:30.398355 24179 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175691491353-13614-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175691491353-13614-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175691491353-13614-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:02:30.414276 24179 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:02:30.416625 24179 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:02:30.416744 24179 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:02:30.416945 24179 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 15:02:30.417294 24179 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 15:02:30.417327 24179 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 15:02:30.417375 24179 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 15:02:30.417402 24179 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 15:02:30.423331 24179 rpc_server.cc:307] RPC server started. Bound to: 127.13.75.131:41685
I20260923 15:02:30.423409 24307 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.75.131:41685 every 8 connection(s)
I20260923 15:02:30.423756 24179 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175691491353-13614-0/minicluster-data/ts-2/data/info.pb
Sep 23 15:02:30 dist-test-slave-cnb0 krb5kdc[23803](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175750, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.131@KRBTEST.COM for kudu/127.13.75.190@KRBTEST.COM
I20260923 15:02:30.430723 13614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKC4soy/build/release/bin/kudu as pid 24179
I20260923 15:02:30.430819 13614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175691491353-13614-0/minicluster-data/ts-2/wal/instance
I20260923 15:02:30.431043 13614 external_mini_cluster.cc:1428] Setting key 24c87d321cd83b50b1f4ecfb233f97ff
I20260923 15:02:30.434141 23887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:30.424556 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.131:48651 (local address 127.13.75.190:38207)
0923 15:02:30.424625 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0923 15:02:30.424627 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:02:30.424859 (+ 232us) server_negotiation.cc:407] Connection header received
0923 15:02:30.425433 (+ 574us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:02:30.425434 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:02:30.425447 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:02:30.425494 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:02:30.426460 (+ 966us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:30.427042 (+ 582us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:30.428064 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:30.428272 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:30.428357 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:30.430828 (+ 2471us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:02:30.430830 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:02:30.430830 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:02:30.430841 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:02:30.432661 (+ 1820us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:02:30.433108 (+ 447us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:02:30.433110 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:02:30.433111 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:02:30.433177 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:02:30.433448 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:02:30.433449 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:02:30.433450 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:02:30.433583 (+ 133us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:02:30.433662 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0923 15:02:30.433952 (+ 290us) server_negotiation.cc:299] Negotiation successful
0923 15:02:30.434039 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260923 15:02:30.434362 24310 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:30.424542 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:38207 (local address 127.13.75.131:48651)
0923 15:02:30.424733 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0923 15:02:30.424772 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0923 15:02:30.425319 (+ 547us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:02:30.425560 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:02:30.425565 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:02:30.426160 (+ 595us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:02:30.426357 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:30.426362 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:30.427136 (+ 774us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:30.427137 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:30.427946 (+ 809us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:30.427948 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:30.428370 (+ 422us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:30.428371 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:30.428504 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:30.429065 (+ 561us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:02:30.429074 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:02:30.430638 (+ 1564us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:02:30.432758 (+ 2120us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:02:30.432766 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:02:30.432768 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:02:30.433015 (+ 247us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:02:30.433246 (+ 231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:02:30.433247 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:02:30.433247 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:02:30.433372 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:02:30.433651 (+ 279us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:02:30.433658 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:02:30.433813 (+ 155us) client_negotiation.cc:769] Sending connection context
0923 15:02:30.433930 (+ 117us) client_negotiation.cc:240] Negotiation successful
0923 15:02:30.434112 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":152,"thread_start_us":98,"threads_started":1}
I20260923 15:02:30.434955 24308 heartbeater.cc:347] Connected to a master server at 127.13.75.190:38207
I20260923 15:02:30.435056 24308 heartbeater.cc:464] Registering TS with master...
I20260923 15:02:30.435213 24308 heartbeater.cc:511] Master 127.13.75.190:38207 requested a full tablet report, sending...
I20260923 15:02:30.435555 23848 ts_manager.cc:194] Registered new tserver with Master: 3e66c62e030e43de816586cf5aee513c (127.13.75.131:41685)
I20260923 15:02:30.435945 23848 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.75.131@KRBTEST.COM'} at 127.13.75.131:48651
I20260923 15:02:30.443782 13614 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260923 15:02:30.452193 23887 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:30.445205 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58760 (local address 127.13.75.190:38207)
0923 15:02:30.445275 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0923 15:02:30.445277 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:02:30.445337 (+ 60us) server_negotiation.cc:407] Connection header received
0923 15:02:30.445396 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:02:30.445397 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:02:30.445406 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:02:30.445461 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:02:30.446107 (+ 646us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:30.446640 (+ 533us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:30.447495 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:30.447704 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:30.447776 (+ 72us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:30.449028 (+ 1252us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:02:30.449029 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:02:30.449030 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:02:30.449037 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:02:30.450586 (+ 1549us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:02:30.451114 (+ 528us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:02:30.451116 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:02:30.451116 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:02:30.451197 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:02:30.451559 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:02:30.451561 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:02:30.451561 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:02:30.451739 (+ 178us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:02:30.451834 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0923 15:02:30.452028 (+ 194us) server_negotiation.cc:299] Negotiation successful
0923 15:02:30.452103 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260923 15:02:30.454300 23848 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58760:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 15:02:30.456192 23848 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 15:02:30.463999 24324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:30.459615 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60476 (local address 127.13.75.130:45175)
0923 15:02:30.459808 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0923 15:02:30.459812 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 15:02:30.459823 (+ 11us) server_negotiation.cc:407] Connection header received
0923 15:02:30.459858 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:02:30.459860 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:02:30.459875 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:02:30.459957 (+ 82us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:02:30.460198 (+ 241us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:30.461101 (+ 903us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:30.462859 (+ 1758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:30.463485 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:30.463586 (+ 101us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:30.463657 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0923 15:02:30.463839 (+ 182us) server_negotiation.cc:299] Negotiation successful
0923 15:02:30.463896 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":97,"threads_started":1}
I20260923 15:02:30.463999 24320 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:30.459246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.129:35885 (local address 127.0.0.1:42442)
0923 15:02:30.459498 (+ 252us) negotiation.cc:107] Waiting for socket to connect
0923 15:02:30.459509 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 15:02:30.459566 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:02:30.460237 (+ 671us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:02:30.460239 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:02:30.460241 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:02:30.460296 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:30.460297 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:30.461757 (+ 1460us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:30.461758 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:30.462667 (+ 909us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:30.462670 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:30.463674 (+ 1004us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:30.463675 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:30.463756 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:30.463761 (+ 5us) client_negotiation.cc:769] Sending connection context
0923 15:02:30.463795 (+ 34us) client_negotiation.cc:240] Negotiation successful
0923 15:02:30.463864 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":230,"thread_start_us":100,"threads_started":1}
I20260923 15:02:30.464202 24321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:30.459266 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50372 (local address 127.13.75.131:41685)
0923 15:02:30.459694 (+ 428us) server_negotiation.cc:206] Beginning negotiation
0923 15:02:30.459699 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 15:02:30.459707 (+ 8us) server_negotiation.cc:407] Connection header received
0923 15:02:30.459774 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:02:30.459776 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:02:30.459794 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:02:30.459883 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:02:30.460161 (+ 278us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:30.461461 (+ 1300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:30.462967 (+ 1506us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:30.463634 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:30.463696 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:30.463754 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0923 15:02:30.464052 (+ 298us) server_negotiation.cc:299] Negotiation successful
0923 15:02:30.464099 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":386,"thread_start_us":83,"threads_started":1}
I20260923 15:02:30.464003 24322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:30.459367 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.130:45175 (local address 127.0.0.1:60476)
0923 15:02:30.459641 (+ 274us) negotiation.cc:107] Waiting for socket to connect
0923 15:02:30.459652 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 15:02:30.459703 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:02:30.459986 (+ 283us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:02:30.459987 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:02:30.459989 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:02:30.460108 (+ 119us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:30.460111 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:30.461258 (+ 1147us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:30.461259 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:30.462742 (+ 1483us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:30.462745 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:30.463585 (+ 840us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:30.463586 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:30.463688 (+ 102us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:30.463704 (+ 16us) client_negotiation.cc:769] Sending connection context
0923 15:02:30.463818 (+ 114us) client_negotiation.cc:240] Negotiation successful
0923 15:02:30.463871 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":251,"mutex_wait_us":66,"thread_start_us":106,"threads_started":1}
I20260923 15:02:30.464013 24319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:30.459125 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.131:41685 (local address 127.0.0.1:50372)
0923 15:02:30.459266 (+ 141us) negotiation.cc:107] Waiting for socket to connect
0923 15:02:30.459318 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0923 15:02:30.459449 (+ 131us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:02:30.459964 (+ 515us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:02:30.459968 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:02:30.459971 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:02:30.460064 (+ 93us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:30.460066 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:30.461579 (+ 1513us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:30.461580 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:30.462867 (+ 1287us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:30.462870 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:30.463786 (+ 916us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:30.463787 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:30.463837 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:30.463840 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 15:02:30.463884 (+ 44us) client_negotiation.cc:240] Negotiation successful
0923 15:02:30.463934 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":102,"mutex_wait_us":60,"thread_start_us":78,"threads_started":1}
I20260923 15:02:30.464793 24323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:30.459481 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42442 (local address 127.13.75.129:35885)
0923 15:02:30.459871 (+ 390us) server_negotiation.cc:206] Beginning negotiation
0923 15:02:30.459876 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 15:02:30.459888 (+ 12us) server_negotiation.cc:407] Connection header received
0923 15:02:30.460102 (+ 214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:02:30.460106 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:02:30.460126 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:02:30.460384 (+ 258us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:02:30.460456 (+ 72us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:30.461653 (+ 1197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:30.462775 (+ 1122us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:30.463563 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:30.464386 (+ 823us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:30.464471 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0923 15:02:30.464636 (+ 165us) server_negotiation.cc:299] Negotiation successful
0923 15:02:30.464711 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":331,"thread_start_us":80,"threads_started":1}
I20260923 15:02:30.465309 24242 tablet_service.cc:1467] Processing CreateTablet for tablet 93507f9e8a3b4ba48379a9c400719ed5 (DEFAULT_TABLE table=test-table [id=35eef660eb1e47b882fe57551c4af732]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 15:02:30.465333 24108 tablet_service.cc:1467] Processing CreateTablet for tablet 93507f9e8a3b4ba48379a9c400719ed5 (DEFAULT_TABLE table=test-table [id=35eef660eb1e47b882fe57551c4af732]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 15:02:30.465406 23973 tablet_service.cc:1467] Processing CreateTablet for tablet 93507f9e8a3b4ba48379a9c400719ed5 (DEFAULT_TABLE table=test-table [id=35eef660eb1e47b882fe57551c4af732]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 15:02:30.465679 24242 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 93507f9e8a3b4ba48379a9c400719ed5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:02:30.465719 23973 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 93507f9e8a3b4ba48379a9c400719ed5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:02:30.465719 24108 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 93507f9e8a3b4ba48379a9c400719ed5. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:02:30.468585 24325 tablet_bootstrap.cc:492] T 93507f9e8a3b4ba48379a9c400719ed5 P 3e66c62e030e43de816586cf5aee513c: Bootstrap starting.
I20260923 15:02:30.469353 24327 tablet_bootstrap.cc:492] T 93507f9e8a3b4ba48379a9c400719ed5 P 816ad4b2ce35454f892ed20c7bed3a57: Bootstrap starting.
I20260923 15:02:30.469451 24325 tablet_bootstrap.cc:654] T 93507f9e8a3b4ba48379a9c400719ed5 P 3e66c62e030e43de816586cf5aee513c: Neither blocks nor log segments found. Creating new log.
I20260923 15:02:30.469661 24326 tablet_bootstrap.cc:492] T 93507f9e8a3b4ba48379a9c400719ed5 P 23e192b3e46e4ed690b9b022e5594f44: Bootstrap starting.
I20260923 15:02:30.470080 24325 log.cc:824] T 93507f9e8a3b4ba48379a9c400719ed5 P 3e66c62e030e43de816586cf5aee513c: Log is configured to *not* fsync() on all Append() calls
I20260923 15:02:30.470395 24326 tablet_bootstrap.cc:654] T 93507f9e8a3b4ba48379a9c400719ed5 P 23e192b3e46e4ed690b9b022e5594f44: Neither blocks nor log segments found. Creating new log.
I20260923 15:02:30.471024 24325 tablet_bootstrap.cc:492] T 93507f9e8a3b4ba48379a9c400719ed5 P 3e66c62e030e43de816586cf5aee513c: No bootstrap required, opened a new log
I20260923 15:02:30.471088 24326 log.cc:824] T 93507f9e8a3b4ba48379a9c400719ed5 P 23e192b3e46e4ed690b9b022e5594f44: Log is configured to *not* fsync() on all Append() calls
I20260923 15:02:30.471132 24325 ts_tablet_manager.cc:1397] T 93507f9e8a3b4ba48379a9c400719ed5 P 3e66c62e030e43de816586cf5aee513c: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260923 15:02:30.471139 24327 tablet_bootstrap.cc:654] T 93507f9e8a3b4ba48379a9c400719ed5 P 816ad4b2ce35454f892ed20c7bed3a57: Neither blocks nor log segments found. Creating new log.
I20260923 15:02:30.471679 24327 log.cc:824] T 93507f9e8a3b4ba48379a9c400719ed5 P 816ad4b2ce35454f892ed20c7bed3a57: Log is configured to *not* fsync() on all Append() calls
I20260923 15:02:30.471946 24326 tablet_bootstrap.cc:492] T 93507f9e8a3b4ba48379a9c400719ed5 P 23e192b3e46e4ed690b9b022e5594f44: No bootstrap required, opened a new log
I20260923 15:02:30.472033 24326 ts_tablet_manager.cc:1397] T 93507f9e8a3b4ba48379a9c400719ed5 P 23e192b3e46e4ed690b9b022e5594f44: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260923 15:02:30.472558 24327 tablet_bootstrap.cc:492] T 93507f9e8a3b4ba48379a9c400719ed5 P 816ad4b2ce35454f892ed20c7bed3a57: No bootstrap required, opened a new log
I20260923 15:02:30.472627 24327 ts_tablet_manager.cc:1397] T 93507f9e8a3b4ba48379a9c400719ed5 P 816ad4b2ce35454f892ed20c7bed3a57: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260923 15:02:30.473078 24325 raft_consensus.cc:348] T 93507f9e8a3b4ba48379a9c400719ed5 P 3e66c62e030e43de816586cf5aee513c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e66c62e030e43de816586cf5aee513c" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 41685 } } peers { permanent_uuid: "23e192b3e46e4ed690b9b022e5594f44" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 45175 } } peers { permanent_uuid: "816ad4b2ce35454f892ed20c7bed3a57" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 35885 } }
I20260923 15:02:30.473242 24325 raft_consensus.cc:374] T 93507f9e8a3b4ba48379a9c400719ed5 P 3e66c62e030e43de816586cf5aee513c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:02:30.473285 24325 raft_consensus.cc:729] T 93507f9e8a3b4ba48379a9c400719ed5 P 3e66c62e030e43de816586cf5aee513c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3e66c62e030e43de816586cf5aee513c, State: Initialized, Role: FOLLOWER
I20260923 15:02:30.473433 24325 consensus_queue.cc:260] T 93507f9e8a3b4ba48379a9c400719ed5 P 3e66c62e030e43de816586cf5aee513c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e66c62e030e43de816586cf5aee513c" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 41685 } } peers { permanent_uuid: "23e192b3e46e4ed690b9b022e5594f44" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 45175 } } peers { permanent_uuid: "816ad4b2ce35454f892ed20c7bed3a57" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 35885 } }
I20260923 15:02:30.473744 24308 heartbeater.cc:503] Master 127.13.75.190:38207 was elected leader, sending a full tablet report...
I20260923 15:02:30.473744 24325 ts_tablet_manager.cc:1428] T 93507f9e8a3b4ba48379a9c400719ed5 P 3e66c62e030e43de816586cf5aee513c: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260923 15:02:30.474102 24326 raft_consensus.cc:348] T 93507f9e8a3b4ba48379a9c400719ed5 P 23e192b3e46e4ed690b9b022e5594f44 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e66c62e030e43de816586cf5aee513c" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 41685 } } peers { permanent_uuid: "23e192b3e46e4ed690b9b022e5594f44" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 45175 } } peers { permanent_uuid: "816ad4b2ce35454f892ed20c7bed3a57" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 35885 } }
I20260923 15:02:30.474275 24326 raft_consensus.cc:374] T 93507f9e8a3b4ba48379a9c400719ed5 P 23e192b3e46e4ed690b9b022e5594f44 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:02:30.474319 24326 raft_consensus.cc:729] T 93507f9e8a3b4ba48379a9c400719ed5 P 23e192b3e46e4ed690b9b022e5594f44 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 23e192b3e46e4ed690b9b022e5594f44, State: Initialized, Role: FOLLOWER
I20260923 15:02:30.474445 24326 consensus_queue.cc:260] T 93507f9e8a3b4ba48379a9c400719ed5 P 23e192b3e46e4ed690b9b022e5594f44 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e66c62e030e43de816586cf5aee513c" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 41685 } } peers { permanent_uuid: "23e192b3e46e4ed690b9b022e5594f44" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 45175 } } peers { permanent_uuid: "816ad4b2ce35454f892ed20c7bed3a57" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 35885 } }
I20260923 15:02:30.474745 24326 ts_tablet_manager.cc:1428] T 93507f9e8a3b4ba48379a9c400719ed5 P 23e192b3e46e4ed690b9b022e5594f44: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260923 15:02:30.474635 24327 raft_consensus.cc:348] T 93507f9e8a3b4ba48379a9c400719ed5 P 816ad4b2ce35454f892ed20c7bed3a57 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e66c62e030e43de816586cf5aee513c" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 41685 } } peers { permanent_uuid: "23e192b3e46e4ed690b9b022e5594f44" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 45175 } } peers { permanent_uuid: "816ad4b2ce35454f892ed20c7bed3a57" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 35885 } }
I20260923 15:02:30.474792 24327 raft_consensus.cc:374] T 93507f9e8a3b4ba48379a9c400719ed5 P 816ad4b2ce35454f892ed20c7bed3a57 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:02:30.474825 24327 raft_consensus.cc:729] T 93507f9e8a3b4ba48379a9c400719ed5 P 816ad4b2ce35454f892ed20c7bed3a57 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 816ad4b2ce35454f892ed20c7bed3a57, State: Initialized, Role: FOLLOWER
I20260923 15:02:30.474984 24174 heartbeater.cc:503] Master 127.13.75.190:38207 was elected leader, sending a full tablet report...
I20260923 15:02:30.474995 24327 consensus_queue.cc:260] T 93507f9e8a3b4ba48379a9c400719ed5 P 816ad4b2ce35454f892ed20c7bed3a57 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e66c62e030e43de816586cf5aee513c" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 41685 } } peers { permanent_uuid: "23e192b3e46e4ed690b9b022e5594f44" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 45175 } } peers { permanent_uuid: "816ad4b2ce35454f892ed20c7bed3a57" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 35885 } }
I20260923 15:02:30.475265 24327 ts_tablet_manager.cc:1428] T 93507f9e8a3b4ba48379a9c400719ed5 P 816ad4b2ce35454f892ed20c7bed3a57: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260923 15:02:30.475340 24040 heartbeater.cc:503] Master 127.13.75.190:38207 was elected leader, sending a full tablet report...
I20260923 15:02:30.485857 24332 raft_consensus.cc:482] T 93507f9e8a3b4ba48379a9c400719ed5 P 23e192b3e46e4ed690b9b022e5594f44 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 15:02:30.486028 24332 raft_consensus.cc:504] T 93507f9e8a3b4ba48379a9c400719ed5 P 23e192b3e46e4ed690b9b022e5594f44 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e66c62e030e43de816586cf5aee513c" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 41685 } } peers { permanent_uuid: "23e192b3e46e4ed690b9b022e5594f44" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 45175 } } peers { permanent_uuid: "816ad4b2ce35454f892ed20c7bed3a57" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 35885 } }
I20260923 15:02:30.486433 24332 leader_election.cc:288] T 93507f9e8a3b4ba48379a9c400719ed5 P 23e192b3e46e4ed690b9b022e5594f44 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 3e66c62e030e43de816586cf5aee513c (127.13.75.131:41685), 816ad4b2ce35454f892ed20c7bed3a57 (127.13.75.129:35885)
I20260923 15:02:30.489828 24321 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:30.486696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.130:38719 (local address 127.13.75.131:41685)
0923 15:02:30.486769 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0923 15:02:30.486771 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:02:30.486784 (+ 13us) server_negotiation.cc:407] Connection header received
0923 15:02:30.486818 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:02:30.486819 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:02:30.486830 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:02:30.486934 (+ 104us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:02:30.487125 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:30.487921 (+ 796us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:30.488984 (+ 1063us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:30.489450 (+ 466us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:30.489509 (+ 59us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:30.489517 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0923 15:02:30.489712 (+ 195us) server_negotiation.cc:299] Negotiation successful
0923 15:02:30.489754 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260923 15:02:30.489820 24176 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:30.486639 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.131:41685 (local address 127.13.75.130:38719)
0923 15:02:30.486699 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0923 15:02:30.486711 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 15:02:30.486756 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:02:30.486952 (+ 196us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:02:30.486953 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:02:30.486960 (+ 7us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:02:30.487041 (+ 81us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:30.487043 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:30.487995 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:30.487996 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:30.488885 (+ 889us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:30.488889 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:30.489532 (+ 643us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:30.489533 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:30.489613 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:30.489617 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 15:02:30.489683 (+ 66us) client_negotiation.cc:240] Negotiation successful
0923 15:02:30.489731 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":38,"mutex_wait_us":29}
I20260923 15:02:30.490000 24334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:30.486763 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.129:35885 (local address 127.13.75.130:54749)
0923 15:02:30.486975 (+ 212us) negotiation.cc:107] Waiting for socket to connect
0923 15:02:30.486985 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 15:02:30.487039 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:02:30.487164 (+ 125us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:02:30.487165 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:02:30.487166 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:02:30.487226 (+ 60us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:30.487228 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:30.488247 (+ 1019us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:30.488248 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:30.489064 (+ 816us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:02:30.489068 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:02:30.489740 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:02:30.489741 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:02:30.489809 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:30.489813 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 15:02:30.489874 (+ 61us) client_negotiation.cc:240] Negotiation successful
0923 15:02:30.489913 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":189,"thread_start_us":113,"threads_started":1}
I20260923 15:02:30.490021 24323 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:30.486955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.130:54749 (local address 127.13.75.129:35885)
0923 15:02:30.487000 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0923 15:02:30.487003 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:02:30.487032 (+ 29us) server_negotiation.cc:407] Connection header received
0923 15:02:30.487087 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:02:30.487088 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:02:30.487097 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:02:30.487156 (+ 59us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:02:30.487435 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:30.488152 (+ 717us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:30.489157 (+ 1005us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:30.489664 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:30.489728 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:30.489736 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0923 15:02:30.489899 (+ 163us) server_negotiation.cc:299] Negotiation successful
0923 15:02:30.489946 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260923 15:02:30.490294 23993 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "93507f9e8a3b4ba48379a9c400719ed5" candidate_uuid: "23e192b3e46e4ed690b9b022e5594f44" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "816ad4b2ce35454f892ed20c7bed3a57" is_pre_election: true
I20260923 15:02:30.490294 24262 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "93507f9e8a3b4ba48379a9c400719ed5" candidate_uuid: "23e192b3e46e4ed690b9b022e5594f44" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e66c62e030e43de816586cf5aee513c" is_pre_election: true
I20260923 15:02:30.490471 24262 raft_consensus.cc:2454] T 93507f9e8a3b4ba48379a9c400719ed5 P 3e66c62e030e43de816586cf5aee513c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 23e192b3e46e4ed690b9b022e5594f44 in term 0.
I20260923 15:02:30.490471 23993 raft_consensus.cc:2454] T 93507f9e8a3b4ba48379a9c400719ed5 P 816ad4b2ce35454f892ed20c7bed3a57 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 23e192b3e46e4ed690b9b022e5594f44 in term 0.
I20260923 15:02:30.490701 24061 leader_election.cc:302] T 93507f9e8a3b4ba48379a9c400719ed5 P 23e192b3e46e4ed690b9b022e5594f44 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 23e192b3e46e4ed690b9b022e5594f44, 816ad4b2ce35454f892ed20c7bed3a57; no voters:
I20260923 15:02:30.490914 24332 raft_consensus.cc:2781] T 93507f9e8a3b4ba48379a9c400719ed5 P 23e192b3e46e4ed690b9b022e5594f44 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 15:02:30.490969 24332 raft_consensus.cc:482] T 93507f9e8a3b4ba48379a9c400719ed5 P 23e192b3e46e4ed690b9b022e5594f44 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 15:02:30.490990 24332 raft_consensus.cc:3037] T 93507f9e8a3b4ba48379a9c400719ed5 P 23e192b3e46e4ed690b9b022e5594f44 [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:02:30.491699 24332 raft_consensus.cc:504] T 93507f9e8a3b4ba48379a9c400719ed5 P 23e192b3e46e4ed690b9b022e5594f44 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e66c62e030e43de816586cf5aee513c" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 41685 } } peers { permanent_uuid: "23e192b3e46e4ed690b9b022e5594f44" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 45175 } } peers { permanent_uuid: "816ad4b2ce35454f892ed20c7bed3a57" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 35885 } }
I20260923 15:02:30.491863 24332 leader_election.cc:288] T 93507f9e8a3b4ba48379a9c400719ed5 P 23e192b3e46e4ed690b9b022e5594f44 [CANDIDATE]: Term 1 election: Requested vote from peers 3e66c62e030e43de816586cf5aee513c (127.13.75.131:41685), 816ad4b2ce35454f892ed20c7bed3a57 (127.13.75.129:35885)
I20260923 15:02:30.492008 24262 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "93507f9e8a3b4ba48379a9c400719ed5" candidate_uuid: "23e192b3e46e4ed690b9b022e5594f44" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "3e66c62e030e43de816586cf5aee513c"
I20260923 15:02:30.492100 23993 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "93507f9e8a3b4ba48379a9c400719ed5" candidate_uuid: "23e192b3e46e4ed690b9b022e5594f44" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "816ad4b2ce35454f892ed20c7bed3a57"
I20260923 15:02:30.492117 24262 raft_consensus.cc:3037] T 93507f9e8a3b4ba48379a9c400719ed5 P 3e66c62e030e43de816586cf5aee513c [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:02:30.492169 23993 raft_consensus.cc:3037] T 93507f9e8a3b4ba48379a9c400719ed5 P 816ad4b2ce35454f892ed20c7bed3a57 [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:02:30.493042 23993 raft_consensus.cc:2454] T 93507f9e8a3b4ba48379a9c400719ed5 P 816ad4b2ce35454f892ed20c7bed3a57 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 23e192b3e46e4ed690b9b022e5594f44 in term 1.
I20260923 15:02:30.493048 24262 raft_consensus.cc:2454] T 93507f9e8a3b4ba48379a9c400719ed5 P 3e66c62e030e43de816586cf5aee513c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 23e192b3e46e4ed690b9b022e5594f44 in term 1.
I20260923 15:02:30.493265 24063 leader_election.cc:302] T 93507f9e8a3b4ba48379a9c400719ed5 P 23e192b3e46e4ed690b9b022e5594f44 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 23e192b3e46e4ed690b9b022e5594f44, 3e66c62e030e43de816586cf5aee513c; no voters:
I20260923 15:02:30.493374 24332 raft_consensus.cc:2781] T 93507f9e8a3b4ba48379a9c400719ed5 P 23e192b3e46e4ed690b9b022e5594f44 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 15:02:30.493541 24332 raft_consensus.cc:686] T 93507f9e8a3b4ba48379a9c400719ed5 P 23e192b3e46e4ed690b9b022e5594f44 [term 1 LEADER]: Becoming Leader. State: Replica: 23e192b3e46e4ed690b9b022e5594f44, State: Running, Role: LEADER
I20260923 15:02:30.493642 24332 consensus_queue.cc:237] T 93507f9e8a3b4ba48379a9c400719ed5 P 23e192b3e46e4ed690b9b022e5594f44 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e66c62e030e43de816586cf5aee513c" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 41685 } } peers { permanent_uuid: "23e192b3e46e4ed690b9b022e5594f44" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 45175 } } peers { permanent_uuid: "816ad4b2ce35454f892ed20c7bed3a57" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 35885 } }
I20260923 15:02:30.494570 23848 catalog_manager.cc:5495] T 93507f9e8a3b4ba48379a9c400719ed5 P 23e192b3e46e4ed690b9b022e5594f44 reported cstate change: term changed from 0 to 1, leader changed from <none> to 23e192b3e46e4ed690b9b022e5594f44 (127.13.75.130). New cstate: current_term: 1 leader_uuid: "23e192b3e46e4ed690b9b022e5594f44" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "3e66c62e030e43de816586cf5aee513c" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 41685 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "23e192b3e46e4ed690b9b022e5594f44" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 45175 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "816ad4b2ce35454f892ed20c7bed3a57" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 35885 } health_report { overall_health: UNKNOWN } } }
I20260923 15:02:30.500142 24324 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:30.497400 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60480 (local address 127.13.75.130:45175)
0923 15:02:30.497458 (+ 58us) server_negotiation.cc:206] Beginning negotiation
0923 15:02:30.497460 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:02:30.497470 (+ 10us) server_negotiation.cc:407] Connection header received
0923 15:02:30.497507 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:02:30.497508 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:02:30.497534 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:02:30.497639 (+ 105us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 15:02:30.497811 (+ 172us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:30.498368 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:30.499290 (+ 922us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:02:30.499503 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:02:30.499595 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:02:30.499770 (+ 175us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 15:02:30.499883 (+ 113us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 15:02:30.499970 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0923 15:02:30.500010 (+ 40us) server_negotiation.cc:299] Negotiation successful
0923 15:02:30.500033 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260923 15:02:30.502128 23993 raft_consensus.cc:1261] T 93507f9e8a3b4ba48379a9c400719ed5 P 816ad4b2ce35454f892ed20c7bed3a57 [term 1 FOLLOWER]: Refusing update from remote peer 23e192b3e46e4ed690b9b022e5594f44: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 15:02:30.502141 24262 raft_consensus.cc:1261] T 93507f9e8a3b4ba48379a9c400719ed5 P 3e66c62e030e43de816586cf5aee513c [term 1 FOLLOWER]: Refusing update from remote peer 23e192b3e46e4ed690b9b022e5594f44: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 15:02:30.502549 24332 consensus_queue.cc:1035] T 93507f9e8a3b4ba48379a9c400719ed5 P 23e192b3e46e4ed690b9b022e5594f44 [LEADER]: Connected to new peer: Peer: permanent_uuid: "816ad4b2ce35454f892ed20c7bed3a57" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 35885 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 15:02:30.502694 24335 consensus_queue.cc:1035] T 93507f9e8a3b4ba48379a9c400719ed5 P 23e192b3e46e4ed690b9b022e5594f44 [LEADER]: Connected to new peer: Peer: permanent_uuid: "3e66c62e030e43de816586cf5aee513c" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 41685 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 15:02:30.505643 24338 mvcc.cc:205] Tried to move back new op lower bound from 7332559874053844992 to 7332559874022625280. Current Snapshot: MvccSnapshot[applied={T|T < 7332559874053844992}]
I20260923 15:02:30.505788 24339 mvcc.cc:205] Tried to move back new op lower bound from 7332559874053844992 to 7332559874022625280. Current Snapshot: MvccSnapshot[applied={T|T < 7332559874053844992}]
I20260923 15:02:30.508064 24341 mvcc.cc:205] Tried to move back new op lower bound from 7332559874053844992 to 7332559874022625280. Current Snapshot: MvccSnapshot[applied={T|T < 7332559874053844992}]
I20260923 15:02:30.509456 23847 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58760:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 15:02:30.509569 23847 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:58760:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 15:02:30.510389 23847 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 15fcd72280b64a03b72ad34341772115: Sending DeleteTablet for 3 replicas of tablet 93507f9e8a3b4ba48379a9c400719ed5
I20260923 15:02:30.510771 24242 tablet_service.cc:1514] Processing DeleteTablet for tablet 93507f9e8a3b4ba48379a9c400719ed5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 15:02:30 UTC) from {username='kudu', principal='kudu/127.13.75.190@KRBTEST.COM'} at 127.0.0.1:50372
I20260923 15:02:30.510761 24108 tablet_service.cc:1514] Processing DeleteTablet for tablet 93507f9e8a3b4ba48379a9c400719ed5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 15:02:30 UTC) from {username='kudu', principal='kudu/127.13.75.190@KRBTEST.COM'} at 127.0.0.1:60476
I20260923 15:02:30.511264 13614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKC4soy/build/release/bin/kudu with pid 23910
I20260923 15:02:30.511307 23973 tablet_service.cc:1514] Processing DeleteTablet for tablet 93507f9e8a3b4ba48379a9c400719ed5 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 15:02:30 UTC) from {username='kudu', principal='kudu/127.13.75.190@KRBTEST.COM'} at 127.0.0.1:42442
I20260923 15:02:30.511919 24348 tablet_replica.cc:330] T 93507f9e8a3b4ba48379a9c400719ed5 P 3e66c62e030e43de816586cf5aee513c: stopping tablet replica
I20260923 15:02:30.512028 24348 raft_consensus.cc:2229] T 93507f9e8a3b4ba48379a9c400719ed5 P 3e66c62e030e43de816586cf5aee513c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 15:02:30.512136 24348 raft_consensus.cc:2258] T 93507f9e8a3b4ba48379a9c400719ed5 P 3e66c62e030e43de816586cf5aee513c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 15:02:30.512307 24349 tablet_replica.cc:330] T 93507f9e8a3b4ba48379a9c400719ed5 P 23e192b3e46e4ed690b9b022e5594f44: stopping tablet replica
I20260923 15:02:30.512426 24349 raft_consensus.cc:2229] T 93507f9e8a3b4ba48379a9c400719ed5 P 23e192b3e46e4ed690b9b022e5594f44 [term 1 LEADER]: Raft consensus shutting down.
I20260923 15:02:30.512528 24349 raft_consensus.cc:2258] T 93507f9e8a3b4ba48379a9c400719ed5 P 23e192b3e46e4ed690b9b022e5594f44 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 15:02:30.512913 24349 ts_tablet_manager.cc:1905] T 93507f9e8a3b4ba48379a9c400719ed5 P 23e192b3e46e4ed690b9b022e5594f44: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 15:02:30.514652 24348 ts_tablet_manager.cc:1905] T 93507f9e8a3b4ba48379a9c400719ed5 P 3e66c62e030e43de816586cf5aee513c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 15:02:30.515012 24349 ts_tablet_manager.cc:1918] T 93507f9e8a3b4ba48379a9c400719ed5 P 23e192b3e46e4ed690b9b022e5594f44: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 15:02:30.515151 24349 log.cc:1196] T 93507f9e8a3b4ba48379a9c400719ed5 P 23e192b3e46e4ed690b9b022e5594f44: Deleting WAL directory at /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175691491353-13614-0/minicluster-data/ts-1/wal/wals/93507f9e8a3b4ba48379a9c400719ed5
I20260923 15:02:30.515528 24349 ts_tablet_manager.cc:1939] T 93507f9e8a3b4ba48379a9c400719ed5 P 23e192b3e46e4ed690b9b022e5594f44: Deleting consensus metadata
I20260923 15:02:30.516044 23834 catalog_manager.cc:4841] TS 23e192b3e46e4ed690b9b022e5594f44 (127.13.75.130:45175): tablet 93507f9e8a3b4ba48379a9c400719ed5 (table test-table [id=35eef660eb1e47b882fe57551c4af732]) successfully deleted
I20260923 15:02:30.516429 24348 ts_tablet_manager.cc:1918] T 93507f9e8a3b4ba48379a9c400719ed5 P 3e66c62e030e43de816586cf5aee513c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 15:02:30.516525 24348 log.cc:1196] T 93507f9e8a3b4ba48379a9c400719ed5 P 3e66c62e030e43de816586cf5aee513c: Deleting WAL directory at /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestRequireAuthenticationSecureCluster.1790175691491353-13614-0/minicluster-data/ts-2/wal/wals/93507f9e8a3b4ba48379a9c400719ed5
I20260923 15:02:30.516768 24348 ts_tablet_manager.cc:1939] T 93507f9e8a3b4ba48379a9c400719ed5 P 3e66c62e030e43de816586cf5aee513c: Deleting consensus metadata
I20260923 15:02:30.517305 23834 catalog_manager.cc:4841] TS 3e66c62e030e43de816586cf5aee513c (127.13.75.131:41685): tablet 93507f9e8a3b4ba48379a9c400719ed5 (table test-table [id=35eef660eb1e47b882fe57551c4af732]) successfully deleted
W20260923 15:02:30.518563 23832 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.75.129:35885 (error 108)
I20260923 15:02:30.518761 13614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKC4soy/build/release/bin/kudu with pid 24045
I20260923 15:02:30.520705 24322 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:02:30.520559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.129:35885 (local address 127.0.0.1:42452)
0923 15:02:30.520626 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0923 15:02:30.520672 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.129:35885: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
W20260923 15:02:30.520859 23832 catalog_manager.cc:4584] TS 816ad4b2ce35454f892ed20c7bed3a57 (127.13.75.129:35885): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 93507f9e8a3b4ba48379a9c400719ed5: Network error: Client connection negotiation failed: client connection to 127.13.75.129:35885: connect: Connection refused (error 111)
I20260923 15:02:30.524540 13614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKC4soy/build/release/bin/kudu with pid 24179
I20260923 15:02:30.529131 13614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKC4soy/build/release/bin/kudu with pid 23816
2026-09-23T15:02:30Z chronyd exiting
[ OK ] SecurityITest.TestRequireAuthenticationSecureCluster (1972 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegration
Loading random data
Initializing database '/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175691491353-13614-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 15:02:30 dist-test-slave-cnb0 krb5kdc[24353](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 15:02:30 dist-test-slave-cnb0 krb5kdc[24353](info): set up 2 sockets
Sep 23 15:02:30 dist-test-slave-cnb0 krb5kdc[24353](info): commencing operation
krb5kdc: starting...
W20260923 15:02:31.617931 13614 mini_kdc.cc:121] Time spent starting KDC: real 1.068s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175691491353-13614-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175691491353-13614-0/krb5kdc/test-user.keytab.
Sep 23 15:02:31 dist-test-slave-cnb0 krb5kdc[24353](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175751, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T15:02:31Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T15:02:31Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.13.75.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.13.75.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.13.75.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175691491353-13614-0/krb5kdc/rangeradmin_127.13.75.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.13.75.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175691491353-13614-0/krb5kdc/rangeradmin_127.13.75.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.13.75.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.13.75.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.13.75.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175691491353-13614-0/krb5kdc/rangerlookup_127.13.75.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.13.75.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175691491353-13614-0/krb5kdc/rangerlookup_127.13.75.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.13.75.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.13.75.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.13.75.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175691491353-13614-0/krb5kdc/HTTP_127.13.75.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.13.75.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175691491353-13614-0/krb5kdc/HTTP_127.13.75.148@KRBTEST.COM.keytab.
I20260923 15:02:31.826848 13614 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175691491353-13614-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskKC4soy/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175691491353-13614-0/minicluster-data/postgres -l logfile start
2026-09-23 15:02:35.084 UTC [24383] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-09-23 15:02:35.084 UTC [24383] LOG: listening on IPv4 address "127.13.75.148", port 58375
2026-09-23 15:02:35.090 UTC [24383] LOG: listening on Unix socket "/tmp/.s.PGSQL.58375"
2026-09-23 15:02:35.127 UTC [24386] LOG: database system was shut down at 2026-09-23 15:02:33 UTC
2026-09-23 15:02:35.132 UTC [24383] LOG: database system is ready to accept connections
I20260923 15:02:36.085002 13614 mini_postgres.cc:96] Postgres bound to 58375
2026-09-23 15:02:36.093 UTC [24394] FATAL: database "slave" does not exist
127.13.75.148:58375 - accepting connections
I20260923 15:02:36.094323 13614 mini_ranger.cc:161] Starting Ranger...
I20260923 15:02:36.113418 13614 mini_ranger.cc:84] Created miniranger Postgres user
I20260923 15:02:36.758976 13614 mini_ranger.cc:87] Created ranger Postgres database
I20260923 15:02:36.759047 13614 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175691491353-13614-0/minicluster-data/ranger-admin
2026-09-23 15:02:37,288 [I] DB FLAVOR :POSTGRES
2026-09-23 15:02:37,289 [I] --------- Verifying Ranger DB connection ---------
2026-09-23 15:02:37,289 [I] Checking connection..
2026-09-23 15:02:37,289 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKC4soy/build/release/bin/postgresql.jar:/tmp/dist-test-taskKC4soy/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.75.148:58375/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-09-23 15:02:37,618 [I] Checking connection passed.
2026-09-23 15:02:37,618 [I] --------- Verifying version history table ---------
2026-09-23 15:02:37,618 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKC4soy/build/release/bin/postgresql.jar:/tmp/dist-test-taskKC4soy/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.75.148:58375/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-23 15:02:37,897 [I] Table x_db_version_h does not exist in database ranger
2026-09-23 15:02:37,897 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKC4soy/build/release/bin/postgresql.jar:/tmp/dist-test-taskKC4soy/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.75.148:58375/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-23 15:02:38,152 [I] Table x_db_version_h does not exist in database ranger
2026-09-23 15:02:38,153 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-09-23 15:02:38,153 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKC4soy/build/release/bin/postgresql.jar:/tmp/dist-test-taskKC4soy/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.75.148:58375/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskKC4soy/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-09-23 15:02:38.423 UTC [24458] WARNING: there is no transaction in progress
2026-09-23 15:02:38,431 [I] create_dbversion_catalog.sql file imported successfully
2026-09-23 15:02:38,431 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKC4soy/build/release/bin/postgresql.jar:/tmp/dist-test-taskKC4soy/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.75.148:58375/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-23 15:02:38,686 [I] Table x_db_version_h already exists in database 'ranger'
2026-09-23 15:02:38,686 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-23 15:02:38,687 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKC4soy/build/release/bin/postgresql.jar:/tmp/dist-test-taskKC4soy/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.75.148:58375/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-09-23 15:02:38,956 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKC4soy/build/release/bin/postgresql.jar:/tmp/dist-test-taskKC4soy/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.75.148:58375/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-09-23 15:02:39,240 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKC4soy/build/release/bin/postgresql.jar:/tmp/dist-test-taskKC4soy/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.75.148:58375/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-cnb0.c.gcp-upstream.internal','N') ;"
2026-09-23 15:02:39,514 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKC4soy/build/release/bin/postgresql.jar:/tmp/dist-test-taskKC4soy/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.75.148:58375/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-23 15:02:39,764 [I] Table x_portal_user does not exist in database ranger
2026-09-23 15:02:39,765 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKC4soy/build/release/bin/postgresql.jar:/tmp/dist-test-taskKC4soy/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.75.148:58375/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-23 15:02:40,033 [I] Table x_policy_ref_group does not exist in database ranger
2026-09-23 15:02:40,033 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKC4soy/build/release/bin/postgresql.jar:/tmp/dist-test-taskKC4soy/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.75.148:58375/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-23 15:02:40,284 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKC4soy/build/release/bin/postgresql.jar:/tmp/dist-test-taskKC4soy/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.75.148:58375/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-23 15:02:40,561 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-09-23 15:02:40,561 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKC4soy/build/release/bin/postgresql.jar:/tmp/dist-test-taskKC4soy/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.75.148:58375/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskKC4soy/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-09-23 15:02:41.693 UTC [24566] WARNING: there is no transaction in progress
2026-09-23 15:02:41.712 UTC [24566] WARNING: there is no transaction in progress
2026-09-23 15:02:41.731 UTC [24566] WARNING: there is no transaction in progress
2026-09-23 15:02:41.749 UTC [24566] WARNING: there is no transaction in progress
2026-09-23 15:02:41.768 UTC [24566] WARNING: there is no transaction in progress
2026-09-23 15:02:41.790 UTC [24566] WARNING: there is no transaction in progress
2026-09-23 15:02:41.860 UTC [24566] WARNING: there is no transaction in progress
2026-09-23 15:02:41.868 UTC [24566] WARNING: there is no transaction in progress
2026-09-23 15:02:41.886 UTC [24566] WARNING: there is no transaction in progress
2026-09-23 15:02:41.898 UTC [24566] WARNING: there is no transaction in progress
2026-09-23 15:02:41.912 UTC [24566] WARNING: there is no transaction in progress
2026-09-23 15:02:41.929 UTC [24566] WARNING: there is no transaction in progress
2026-09-23 15:02:41.942 UTC [24566] WARNING: there is no transaction in progress
2026-09-23 15:02:41.950 UTC [24566] WARNING: there is no transaction in progress
2026-09-23 15:02:41.959 UTC [24566] WARNING: there is no transaction in progress
2026-09-23 15:02:41.968 UTC [24566] WARNING: there is no transaction in progress
2026-09-23 15:02:42,806 [I] ranger_core_db_postgres.sql file imported successfully
2026-09-23 15:02:42,807 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKC4soy/build/release/bin/postgresql.jar:/tmp/dist-test-taskKC4soy/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.75.148:58375/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-09-23 15:02:43,054 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-09-23 15:02:43,055 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKC4soy/build/release/bin/postgresql.jar:/tmp/dist-test-taskKC4soy/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.75.148:58375/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-23 15:02:43,333 [I] Table x_portal_user already exists in database 'ranger'
2026-09-23 15:02:43,333 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKC4soy/build/release/bin/postgresql.jar:/tmp/dist-test-taskKC4soy/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.75.148:58375/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-23 15:02:43,618 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-09-23 15:02:43,618 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKC4soy/build/release/bin/postgresql.jar:/tmp/dist-test-taskKC4soy/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.75.148:58375/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-23 15:02:43,891 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKC4soy/build/release/bin/postgresql.jar:/tmp/dist-test-taskKC4soy/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.75.148:58375/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-23 15:02:44,172 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKC4soy/build/release/bin/postgresql.jar:/tmp/dist-test-taskKC4soy/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.75.148:58375/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-cnb0.c.gcp-upstream.internal';"
2026-09-23 15:02:44,449 [I] CORE_DB_SCHEMA import status has been updated
2026-09-23 15:02:44,449 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKC4soy/build/release/bin/postgresql.jar:/tmp/dist-test-taskKC4soy/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.75.148:58375/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-09-23 15:02:44,734 [I] DB_PATCHES have already been applied
I20260923 15:02:44.740767 13614 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175691491353-13614-0/minicluster-data/ranger-admin:/tmp/dist-test-taskKC4soy/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskKC4soy/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskKC4soy/build/release/bin/postgresql.jar:/tmp/dist-test-taskKC4soy/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260923 15:02:44.740839 13614 mini_ranger.cc:193] Using host: 127.13.75.148
I20260923 15:02:44.744043 13614 mini_ranger.cc:227] Ranger admin URL: http://127.13.75.148:47597
Sep 23, 2026 3:02:45 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 23, 2026 3:02:45 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 23, 2026 3:02:45 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175691491353-13614-0/minicluster-data/ranger-admin/ews/webapp
Sep 23, 2026 3:02:45 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175691491353-13614-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Sep 23, 2026 3:02:45 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175691491353-13614-0/minicluster-data/ranger-admin/ews/webapp] .....
Sep 23, 2026 3:02:46 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175691491353-13614-0/minicluster-data/ranger-admin/ews/webapp].
Sep 23, 2026 3:02:46 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260923 15:02:47.006865 13614 mini_ranger.cc:160] Time spent starting Ranger: real 10.913s user 0.000s sys 0.011s
Sep 23, 2026 3:03:08 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Sep 23, 2026 3:03:08 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.XKeyREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.UserREST
Sep 23, 2026 3:03:08 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Sep 23, 2026 3:03:08 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Sep 23, 2026 3:03:08 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Sep 23, 2026 3:03:08 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Sep 23, 2026 3:03:08 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Sep 23, 2026 3:03:08 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Sep 23, 2026 3:03:08 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Sep 23, 2026 3:03:08 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Sep 23, 2026 3:03:08 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Sep 23, 2026 3:03:08 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Sep 23, 2026 3:03:08 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Sep 23, 2026 3:03:08 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Sep 23, 2026 3:03:08 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Sep 23, 2026 3:03:08 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Sep 23, 2026 3:03:08 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Sep 23, 2026 3:03:09 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260923 15:03:10.147594 13614 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.13.75.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.13.75.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.13.75.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175691491353-13614-0/krb5kdc/rangerkms_127.13.75.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.13.75.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175691491353-13614-0/krb5kdc/rangerkms_127.13.75.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.13.75.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.13.75.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.13.75.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175691491353-13614-0/krb5kdc/HTTP_127.13.75.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.13.75.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175691491353-13614-0/krb5kdc/HTTP_127.13.75.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Sep 23 15:03:10 dist-test-slave-cnb0 krb5kdc[24353](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175790, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260923 15:03:10.241788 13614 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260923 15:03:10.252060 13614 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260923 15:03:11.345623 13614 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260923 15:03:11.345713 13614 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175691491353-13614-0/minicluster-data/ranger-kms
I20260923 15:03:11.345733 13614 mini_ranger_kms.cc:227] Using postgres at 127.13.75.148:58375
2026-09-23 15:03:11,970 [I] DB FLAVOR :POSTGRES
2026-09-23 15:03:11,971 [I] --------- Verifying Ranger DB connection ---------
2026-09-23 15:03:11,971 [I] Checking connection
2026-09-23 15:03:11,971 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKC4soy/build/release/bin/postgresql.jar:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175691491353-13614-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.75.148:58375/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-09-23 15:03:12,258 [I] connection success
2026-09-23 15:03:12,258 [I] --------- Verifying Ranger DB tables ---------
2026-09-23 15:03:12,258 [I] Verifying table ranger_masterkey in database rangerkms
2026-09-23 15:03:12,258 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKC4soy/build/release/bin/postgresql.jar:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175691491353-13614-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.75.148:58375/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-09-23 15:03:12,498 [I] Table ranger_masterkey does not exist in database rangerkms
2026-09-23 15:03:12,499 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-23 15:03:12,499 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-09-23 15:03:12,499 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKC4soy/build/release/bin/postgresql.jar:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175691491353-13614-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.75.148:58375/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175691491353-13614-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-09-23 15:03:12,772 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260923 15:03:13.304735 13614 mini_ranger_kms.cc:310] Created kms service
I20260923 15:03:13.514351 13614 mini_ranger_kms.cc:326] Created kudu user
I20260923 15:03:13.561550 13614 mini_ranger_kms.cc:343] Created rangerkms user
Sep 23 15:03:13 dist-test-slave-cnb0 krb5kdc[24353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175790, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.13.75.148@KRBTEST.COM
I20260923 15:03:14.004230 13614 mini_ranger_kms.cc:384] Added ranger policy
I20260923 15:03:14.004354 13614 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175691491353-13614-0/minicluster-data/ranger-kms:/tmp/dist-test-taskKC4soy/build/release/bin/postgresql.jar:/tmp/dist-test-taskKC4soy/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskKC4soy/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260923 15:03:14.004374 13614 mini_ranger_kms.cc:240] Using host: 127.13.75.148
I20260923 15:03:14.008320 13614 mini_ranger_kms.cc:276] Ranger KMS PID: 24750
I20260923 15:03:14.008551 13614 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.13.75.148:58833
Sep 23, 2026 3:03:15 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 23, 2026 3:03:15 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 23, 2026 3:03:15 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175691491353-13614-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Sep 23, 2026 3:03:15 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175691491353-13614-0/minicluster-data/ranger-kms/ews/webapp] .....
Sep 23, 2026 3:03:15 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175691491353-13614-0/minicluster-data/ranger-kms/ews/webapp].
Sep 23, 2026 3:03:15 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
I20260923 15:03:16.223088 13614 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.981s user 0.000s sys 0.246s
I20260923 15:03:16.223204 13614 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260923 15:03:16.223290 13614 mini_ranger_kms.cc:401] 127.13.75.148:58833/kms/v1/keys
Sep 23 15:03:16 dist-test-slave-cnb0 krb5kdc[24353](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790175796, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.13.75.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Sep 23 15:03:16 dist-test-slave-cnb0 krb5kdc[24353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175796, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.13.75.148@KRBTEST.COM for HTTP/127.13.75.148@KRBTEST.COM
Sep 23 15:03:17 dist-test-slave-cnb0 krb5kdc[24353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175796, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.13.75.148@KRBTEST.COM for HTTP/127.13.75.148@KRBTEST.COM
Sep 23 15:03:19 dist-test-slave-cnb0 krb5kdc[24353](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175799, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.13.75.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.75.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.75.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.190.keytab.
Entry for principal kudu/127.13.75.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.190.keytab.
I20260923 15:03:19.595083 13614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKC4soy/build/release/bin/kudu
/tmp/dist-test-taskKC4soy/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175691491353-13614-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175691491353-13614-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175691491353-13614-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175691491353-13614-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.75.190:34207
--webserver_interface=127.13.75.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.190.keytab
--principal=kudu/127.13.75.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.75.148:33553
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.75.190:34207
--ranger_config_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175691491353-13614-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.13.75.148:58833/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175691491353-13614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175691491353-13614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175691491353-13614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:03:19.681468 24800 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:03:19.681706 24800 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:03:19.681743 24800 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:03:19.683027 24800 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 15:03:19.683087 24800 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:03:19.683112 24800 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:03:19.683141 24800 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 15:03:19.683167 24800 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 15:03:19.684741 24800 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.75.148:33553
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175691491353-13614-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175691491353-13614-0/minicluster-data/master-0/wal
--ranger_kms_url=127.13.75.148:58833/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.13.75.190:34207
--ranger_config_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175691491353-13614-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.190.keytab
--principal=kudu/127.13.75.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.75.190:34207
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175691491353-13614-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.75.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24800
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175691491353-13614-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:03:19.685024 24800 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:03:19.685257 24800 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:03:19.691702 24805 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:03:19.691798 24808 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:03:19.691854 24800 server_base.cc:1034] running on GCE node
W20260923 15:03:19.691706 24806 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:03:19.692154 24800 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:03:19.692539 24800 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:03:19.693711 24800 hybrid_clock.cc:648] HybridClock initialized: now 1790175799693698 us; error 29 us; skew 500 ppm
Sep 23 15:03:19 dist-test-slave-cnb0 krb5kdc[24353](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175799, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:03:19.696851 24800 init.cc:377] Logged in from keytab as kudu/127.13.75.190@KRBTEST.COM (short username kudu)
I20260923 15:03:19.697407 24800 webserver.cc:460] Webserver started at http://127.13.75.190:43323/ using document root <none> and password file <none>
I20260923 15:03:19.697664 24800 fs_manager.cc:359] Metadata directory not provided
I20260923 15:03:19.697726 24800 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:03:19.697889 24800 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 23 15:03:19 dist-test-slave-cnb0 krb5kdc[24353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175799, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.190@KRBTEST.COM for HTTP/127.13.75.148@KRBTEST.COM
I20260923 15:03:19.761767 24800 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175691491353-13614-0/minicluster-data/master-0/data/instance:
uuid: "7d6d822f805249e899cd6dd8ba134b0c"
format_stamp: "Formatted at 2026-09-23 15:03:19 on dist-test-slave-cnb0"
server_key: "e5c5f8447ae77775ebb2db53d9afd4e4"
server_key_iv: "dbc4977ab0157403f9df20985d35dc5e"
server_key_version: "kuduclusterkey@0"
I20260923 15:03:19.762336 24800 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175691491353-13614-0/minicluster-data/master-0/wal/instance:
uuid: "7d6d822f805249e899cd6dd8ba134b0c"
format_stamp: "Formatted at 2026-09-23 15:03:19 on dist-test-slave-cnb0"
server_key: "e5c5f8447ae77775ebb2db53d9afd4e4"
server_key_iv: "dbc4977ab0157403f9df20985d35dc5e"
server_key_version: "kuduclusterkey@0"
I20260923 15:03:19.763892 24800 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.001s
I20260923 15:03:19.779091 24815 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:03:19.779282 24800 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:03:19.779397 24800 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175691491353-13614-0/minicluster-data/master-0/data,/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175691491353-13614-0/minicluster-data/master-0/wal
uuid: "7d6d822f805249e899cd6dd8ba134b0c"
format_stamp: "Formatted at 2026-09-23 15:03:19 on dist-test-slave-cnb0"
server_key: "e5c5f8447ae77775ebb2db53d9afd4e4"
server_key_iv: "dbc4977ab0157403f9df20985d35dc5e"
server_key_version: "kuduclusterkey@0"
I20260923 15:03:19.779508 24800 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175691491353-13614-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175691491353-13614-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175691491353-13614-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:03:19.791200 24800 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:03:19.793373 24800 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:03:19.793535 24800 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:03:19.798674 24800 rpc_server.cc:307] RPC server started. Bound to: 127.13.75.190:34207
I20260923 15:03:19.798694 24867 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.75.190:34207 every 8 connection(s)
I20260923 15:03:19.799170 24800 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175691491353-13614-0/minicluster-data/master-0/data/info.pb
I20260923 15:03:19.800021 24868 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:03:19.801898 13614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKC4soy/build/release/bin/kudu as pid 24800
I20260923 15:03:19.802001 13614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175691491353-13614-0/minicluster-data/master-0/wal/instance
I20260923 15:03:19.802186 13614 external_mini_cluster.cc:1428] Setting key cfefd26e50cd5d5fc198f179f385fece
I20260923 15:03:19.802734 24868 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7d6d822f805249e899cd6dd8ba134b0c: Bootstrap starting.
I20260923 15:03:19.803471 24868 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7d6d822f805249e899cd6dd8ba134b0c: Neither blocks nor log segments found. Creating new log.
I20260923 15:03:19.803854 24868 log.cc:824] T 00000000000000000000000000000000 P 7d6d822f805249e899cd6dd8ba134b0c: Log is configured to *not* fsync() on all Append() calls
I20260923 15:03:19.804704 24868 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7d6d822f805249e899cd6dd8ba134b0c: No bootstrap required, opened a new log
I20260923 15:03:19.806988 24868 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7d6d822f805249e899cd6dd8ba134b0c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d6d822f805249e899cd6dd8ba134b0c" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 34207 } }
I20260923 15:03:19.807193 24868 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7d6d822f805249e899cd6dd8ba134b0c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:03:19.807260 24868 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7d6d822f805249e899cd6dd8ba134b0c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7d6d822f805249e899cd6dd8ba134b0c, State: Initialized, Role: FOLLOWER
I20260923 15:03:19.807427 24868 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7d6d822f805249e899cd6dd8ba134b0c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d6d822f805249e899cd6dd8ba134b0c" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 34207 } }
I20260923 15:03:19.807526 24868 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7d6d822f805249e899cd6dd8ba134b0c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 15:03:19.807569 24868 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7d6d822f805249e899cd6dd8ba134b0c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 15:03:19.807621 24868 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7d6d822f805249e899cd6dd8ba134b0c [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:03:19.808634 24868 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7d6d822f805249e899cd6dd8ba134b0c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d6d822f805249e899cd6dd8ba134b0c" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 34207 } }
I20260923 15:03:19.808760 24868 leader_election.cc:302] T 00000000000000000000000000000000 P 7d6d822f805249e899cd6dd8ba134b0c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7d6d822f805249e899cd6dd8ba134b0c; no voters:
I20260923 15:03:19.808921 24868 leader_election.cc:288] T 00000000000000000000000000000000 P 7d6d822f805249e899cd6dd8ba134b0c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 15:03:19.809001 24873 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7d6d822f805249e899cd6dd8ba134b0c [term 1 FOLLOWER]: Leader election won for term 1
I20260923 15:03:19.809180 24873 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7d6d822f805249e899cd6dd8ba134b0c [term 1 LEADER]: Becoming Leader. State: Replica: 7d6d822f805249e899cd6dd8ba134b0c, State: Running, Role: LEADER
I20260923 15:03:19.809283 24868 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7d6d822f805249e899cd6dd8ba134b0c [sys.catalog]: configured and running, proceeding with master startup.
I20260923 15:03:19.809329 24873 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7d6d822f805249e899cd6dd8ba134b0c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d6d822f805249e899cd6dd8ba134b0c" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 34207 } }
I20260923 15:03:19.809723 24873 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7d6d822f805249e899cd6dd8ba134b0c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7d6d822f805249e899cd6dd8ba134b0c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d6d822f805249e899cd6dd8ba134b0c" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 34207 } } }
I20260923 15:03:19.809859 24873 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7d6d822f805249e899cd6dd8ba134b0c [sys.catalog]: This master's current role is: LEADER
I20260923 15:03:19.810070 24873 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7d6d822f805249e899cd6dd8ba134b0c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7d6d822f805249e899cd6dd8ba134b0c. Latest consensus state: current_term: 1 leader_uuid: "7d6d822f805249e899cd6dd8ba134b0c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7d6d822f805249e899cd6dd8ba134b0c" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 34207 } } }
I20260923 15:03:19.810177 24873 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7d6d822f805249e899cd6dd8ba134b0c [sys.catalog]: This master's current role is: LEADER
Sep 23 15:03:19 dist-test-slave-cnb0 krb5kdc[24353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175799, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.75.190@KRBTEST.COM
I20260923 15:03:19.810572 24877 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 15:03:19.811185 24877 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 15:03:19.811918 24868 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175691491353-13614-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260923 15:03:19.816496 24877 catalog_manager.cc:1347] Generated new cluster ID: dfcd51afd75a4fffa3b3f02ce4f8e6f0
I20260923 15:03:19.816574 24877 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 15:03:19.819168 24871 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:03:19.802743 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47440 (local address 127.13.75.190:34207)
0923 15:03:19.802970 (+ 227us) server_negotiation.cc:206] Beginning negotiation
0923 15:03:19.802980 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0923 15:03:19.803013 (+ 33us) server_negotiation.cc:407] Connection header received
0923 15:03:19.803581 (+ 568us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:03:19.803590 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:03:19.803655 (+ 65us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:03:19.803809 (+ 154us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:03:19.804597 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:03:19.805972 (+ 1375us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:03:19.807073 (+ 1101us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:03:19.807680 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:03:19.807787 (+ 107us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:03:19.810762 (+ 2975us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:03:19.810766 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:03:19.810766 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:03:19.810779 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:03:19.814334 (+ 3555us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:03:19.814819 (+ 485us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:03:19.814824 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:03:19.814824 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:03:19.815127 (+ 303us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:03:19.815361 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:03:19.815362 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:03:19.815362 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:03:19.815938 (+ 576us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:03:19.816137 (+ 199us) server_negotiation.cc:1035] Waiting for connection context
0923 15:03:19.816204 (+ 67us) server_negotiation.cc:299] Negotiation successful
0923 15:03:19.816375 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":181,"thread_start_us":81,"threads_started":1}
I20260923 15:03:19.839141 24877 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 15:03:19.840646 24877 catalog_manager.cc:1504] Loading token signing keys...
I20260923 15:03:19.854707 24877 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7d6d822f805249e899cd6dd8ba134b0c: Generated new TSK 0
I20260923 15:03:19.855043 24877 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.75.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.75.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.75.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.129.keytab.
Entry for principal kudu/127.13.75.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.129.keytab.
I20260923 15:03:20.662729 13614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKC4soy/build/release/bin/kudu
/tmp/dist-test-taskKC4soy/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175691491353-13614-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175691491353-13614-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175691491353-13614-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175691491353-13614-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.13.75.148:58833/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.75.129:0
--local_ip_for_outbound_sockets=127.13.75.129
--webserver_interface=127.13.75.129
--webserver_port=0
--tserver_master_addrs=127.13.75.190:34207
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.129.keytab
--principal=kudu/127.13.75.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.75.148:33553
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175691491353-13614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175691491353-13614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175691491353-13614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:03:20.800716 24913 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:03:20.801090 24913 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:03:20.801155 24913 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:03:20.803056 24913 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:03:20.803118 24913 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:03:20.803162 24913 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.75.129
I20260923 15:03:20.805583 24913 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.75.148:33553
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175691491353-13614-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175691491353-13614-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.13.75.148:58833/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.129.keytab
--principal=kudu/127.13.75.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.75.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175691491353-13614-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.75.129
--webserver_port=0
--tserver_master_addrs=127.13.75.190:34207
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.24913
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.75.129
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175691491353-13614-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:03:20.805851 24913 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:03:20.806174 24913 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:03:20.813239 24919 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:03:20.813288 24921 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:03:20.815060 24918 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:03:20.815984 24913 server_base.cc:1034] running on GCE node
I20260923 15:03:20.816224 24913 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:03:20.816506 24913 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:03:20.817688 24913 hybrid_clock.cc:648] HybridClock initialized: now 1790175800817665 us; error 34 us; skew 500 ppm
Sep 23 15:03:20 dist-test-slave-cnb0 krb5kdc[24353](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175800, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:03:20.825263 24913 init.cc:377] Logged in from keytab as kudu/127.13.75.129@KRBTEST.COM (short username kudu)
I20260923 15:03:20.825866 24913 webserver.cc:460] Webserver started at http://127.13.75.129:39945/ using document root <none> and password file <none>
I20260923 15:03:20.826129 24913 fs_manager.cc:359] Metadata directory not provided
I20260923 15:03:20.826184 24913 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:03:20.826313 24913 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 23 15:03:20 dist-test-slave-cnb0 krb5kdc[24353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175800, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.129@KRBTEST.COM for HTTP/127.13.75.148@KRBTEST.COM
I20260923 15:03:20.843165 24913 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175691491353-13614-0/minicluster-data/ts-0/data/instance:
uuid: "73ae947088af44479a73b84f225c691b"
format_stamp: "Formatted at 2026-09-23 15:03:20 on dist-test-slave-cnb0"
server_key: "8e0ffa729f132f4cb3ccaa763f1de46c"
server_key_iv: "74bc38d1a33b1e6baa7d42562463c2a4"
server_key_version: "kuduclusterkey@0"
I20260923 15:03:20.843676 24913 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175691491353-13614-0/minicluster-data/ts-0/wal/instance:
uuid: "73ae947088af44479a73b84f225c691b"
format_stamp: "Formatted at 2026-09-23 15:03:20 on dist-test-slave-cnb0"
server_key: "8e0ffa729f132f4cb3ccaa763f1de46c"
server_key_iv: "74bc38d1a33b1e6baa7d42562463c2a4"
server_key_version: "kuduclusterkey@0"
I20260923 15:03:20.847123 24913 fs_manager.cc:691] Time spent creating directory manager: real 0.003s user 0.001s sys 0.000s
I20260923 15:03:20.865741 24930 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:03:20.865991 24913 fs_manager.cc:725] Time spent opening block manager: real 0.003s user 0.001s sys 0.000s
I20260923 15:03:20.866080 24913 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175691491353-13614-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175691491353-13614-0/minicluster-data/ts-0/wal
uuid: "73ae947088af44479a73b84f225c691b"
format_stamp: "Formatted at 2026-09-23 15:03:20 on dist-test-slave-cnb0"
server_key: "8e0ffa729f132f4cb3ccaa763f1de46c"
server_key_iv: "74bc38d1a33b1e6baa7d42562463c2a4"
server_key_version: "kuduclusterkey@0"
I20260923 15:03:20.866174 24913 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175691491353-13614-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175691491353-13614-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175691491353-13614-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:03:20.888942 24913 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:03:20.891340 24913 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:03:20.891542 24913 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:03:20.891762 24913 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 15:03:20.892170 24913 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 15:03:20.892242 24913 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 15:03:20.892292 24913 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 15:03:20.892328 24913 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 15:03:20.899120 24913 rpc_server.cc:307] RPC server started. Bound to: 127.13.75.129:43021
I20260923 15:03:20.899643 24913 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175691491353-13614-0/minicluster-data/ts-0/data/info.pb
I20260923 15:03:20.907140 13614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKC4soy/build/release/bin/kudu as pid 24913
I20260923 15:03:20.907253 13614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175691491353-13614-0/minicluster-data/ts-0/wal/instance
I20260923 15:03:20.907450 13614 external_mini_cluster.cc:1428] Setting key a425d058b539056699e6805c1537ce46
WARNING: no policy specified for kudu/127.13.75.130@KRBTEST.COM; defaulting to no policy
I20260923 15:03:20.931324 25044 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.75.129:43021 every 8 connection(s)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.75.130@KRBTEST.COM" created.
Sep 23 15:03:20 dist-test-slave-cnb0 krb5kdc[24353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175800, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.129@KRBTEST.COM for kudu/127.13.75.190@KRBTEST.COM
I20260923 15:03:20.943758 25049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:03:20.932474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.129:44139 (local address 127.13.75.190:34207)
0923 15:03:20.933948 (+ 1474us) server_negotiation.cc:206] Beginning negotiation
0923 15:03:20.933951 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:03:20.933960 (+ 9us) server_negotiation.cc:407] Connection header received
0923 15:03:20.933987 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:03:20.933988 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:03:20.934001 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:03:20.934072 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:03:20.934934 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:03:20.935738 (+ 804us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:03:20.937242 (+ 1504us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:03:20.937572 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:03:20.937635 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:03:20.940290 (+ 2655us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:03:20.940292 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:03:20.940292 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:03:20.940300 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:03:20.942148 (+ 1848us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:03:20.942703 (+ 555us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:03:20.942704 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:03:20.942705 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:03:20.942773 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:03:20.943080 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:03:20.943081 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:03:20.943081 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:03:20.943208 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:03:20.943286 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0923 15:03:20.943569 (+ 283us) server_negotiation.cc:299] Negotiation successful
0923 15:03:20.943657 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1449,"thread_start_us":56,"threads_started":1}
I20260923 15:03:20.943958 25048 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:03:20.932098 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:34207 (local address 127.13.75.129:44139)
0923 15:03:20.933020 (+ 922us) negotiation.cc:107] Waiting for socket to connect
0923 15:03:20.933077 (+ 57us) client_negotiation.cc:174] Beginning negotiation
0923 15:03:20.933713 (+ 636us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:03:20.934196 (+ 483us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:03:20.934202 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:03:20.934469 (+ 267us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:03:20.934767 (+ 298us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:03:20.934773 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:03:20.935851 (+ 1078us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:03:20.935853 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:03:20.937139 (+ 1286us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:03:20.937142 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:03:20.937703 (+ 561us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:03:20.937704 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:03:20.937844 (+ 140us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:03:20.938459 (+ 615us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:03:20.938469 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:03:20.940175 (+ 1706us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:03:20.942245 (+ 2070us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:03:20.942253 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:03:20.942255 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:03:20.942601 (+ 346us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:03:20.942843 (+ 242us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:03:20.942844 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:03:20.942844 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:03:20.942999 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:03:20.943275 (+ 276us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:03:20.943282 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:03:20.943434 (+ 152us) client_negotiation.cc:769] Sending connection context
0923 15:03:20.943547 (+ 113us) client_negotiation.cc:240] Negotiation successful
0923 15:03:20.943702 (+ 155us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":867,"thread_start_us":124,"threads_started":1}
I20260923 15:03:20.944641 25045 heartbeater.cc:347] Connected to a master server at 127.13.75.190:34207
I20260923 15:03:20.944761 25045 heartbeater.cc:464] Registering TS with master...
I20260923 15:03:20.945001 25045 heartbeater.cc:511] Master 127.13.75.190:34207 requested a full tablet report, sending...
I20260923 15:03:20.945570 24832 ts_manager.cc:194] Registered new tserver with Master: 73ae947088af44479a73b84f225c691b (127.13.75.129:43021)
I20260923 15:03:20.946439 24832 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.75.129@KRBTEST.COM'} at 127.13.75.129:44139
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.75.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.130.keytab.
Entry for principal kudu/127.13.75.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.130.keytab.
I20260923 15:03:20.965572 13614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKC4soy/build/release/bin/kudu
/tmp/dist-test-taskKC4soy/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175691491353-13614-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175691491353-13614-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175691491353-13614-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175691491353-13614-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.13.75.148:58833/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.75.130:0
--local_ip_for_outbound_sockets=127.13.75.130
--webserver_interface=127.13.75.130
--webserver_port=0
--tserver_master_addrs=127.13.75.190:34207
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.130.keytab
--principal=kudu/127.13.75.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.75.148:33553
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175691491353-13614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175691491353-13614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175691491353-13614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 23 15:03:21 dist-test-slave-cnb0 krb5kdc[24353](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790175801, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260923 15:03:21.064545 25053 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:03:21.064781 25053 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:03:21.064823 25053 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:03:21.066036 25053 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:03:21.066104 25053 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:03:21.066149 25053 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.75.130
I20260923 15:03:21.067732 25053 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.75.148:33553
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175691491353-13614-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175691491353-13614-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.13.75.148:58833/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.130.keytab
--principal=kudu/127.13.75.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.75.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175691491353-13614-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.75.130
--webserver_port=0
--tserver_master_addrs=127.13.75.190:34207
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25053
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.75.130
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175691491353-13614-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:03:21.068013 25053 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:03:21.068257 25053 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:03:21.073779 25058 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:03:21.073884 25061 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:03:21.073884 25059 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:03:21.074548 25053 server_base.cc:1034] running on GCE node
I20260923 15:03:21.074800 25053 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:03:21.075114 25053 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:03:21.076321 25053 hybrid_clock.cc:648] HybridClock initialized: now 1790175801076304 us; error 34 us; skew 500 ppm
Sep 23 15:03:21 dist-test-slave-cnb0 krb5kdc[24353](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175801, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:03:21.079077 25053 init.cc:377] Logged in from keytab as kudu/127.13.75.130@KRBTEST.COM (short username kudu)
I20260923 15:03:21.079651 25053 webserver.cc:460] Webserver started at http://127.13.75.130:36143/ using document root <none> and password file <none>
I20260923 15:03:21.079932 25053 fs_manager.cc:359] Metadata directory not provided
I20260923 15:03:21.079999 25053 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:03:21.080181 25053 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 23 15:03:21 dist-test-slave-cnb0 krb5kdc[24353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175801, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.130@KRBTEST.COM for HTTP/127.13.75.148@KRBTEST.COM
I20260923 15:03:21.097759 25053 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175691491353-13614-0/minicluster-data/ts-1/data/instance:
uuid: "8e58a506d02a4ceb9b4633414637bd44"
format_stamp: "Formatted at 2026-09-23 15:03:21 on dist-test-slave-cnb0"
server_key: "fb713f73a34c741213548c9235c61a99"
server_key_iv: "4412a6f4fb1f294b6cab7a6ad9f1eacf"
server_key_version: "kuduclusterkey@0"
I20260923 15:03:21.098333 25053 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175691491353-13614-0/minicluster-data/ts-1/wal/instance:
uuid: "8e58a506d02a4ceb9b4633414637bd44"
format_stamp: "Formatted at 2026-09-23 15:03:21 on dist-test-slave-cnb0"
server_key: "fb713f73a34c741213548c9235c61a99"
server_key_iv: "4412a6f4fb1f294b6cab7a6ad9f1eacf"
server_key_version: "kuduclusterkey@0"
I20260923 15:03:21.115525 25053 fs_manager.cc:691] Time spent creating directory manager: real 0.017s user 0.002s sys 0.000s
I20260923 15:03:21.136963 25069 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:03:21.137272 25053 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:03:21.137431 25053 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175691491353-13614-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175691491353-13614-0/minicluster-data/ts-1/wal
uuid: "8e58a506d02a4ceb9b4633414637bd44"
format_stamp: "Formatted at 2026-09-23 15:03:21 on dist-test-slave-cnb0"
server_key: "fb713f73a34c741213548c9235c61a99"
server_key_iv: "4412a6f4fb1f294b6cab7a6ad9f1eacf"
server_key_version: "kuduclusterkey@0"
I20260923 15:03:21.137621 25053 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175691491353-13614-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175691491353-13614-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175691491353-13614-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:03:21.166325 25053 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:03:21.168545 25053 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:03:21.168730 25053 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:03:21.169013 25053 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 15:03:21.169409 25053 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 15:03:21.169472 25053 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 15:03:21.169553 25053 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 15:03:21.169584 25053 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 15:03:21.177304 25053 rpc_server.cc:307] RPC server started. Bound to: 127.13.75.130:37027
I20260923 15:03:21.177795 25053 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175691491353-13614-0/minicluster-data/ts-1/data/info.pb
I20260923 15:03:21.179324 25182 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.75.130:37027 every 8 connection(s)
I20260923 15:03:21.183776 13614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKC4soy/build/release/bin/kudu as pid 25053
I20260923 15:03:21.183893 13614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175691491353-13614-0/minicluster-data/ts-1/wal/instance
I20260923 15:03:21.184093 13614 external_mini_cluster.cc:1428] Setting key d15b155989665e38397ea6b81fec30b3
Sep 23 15:03:21 dist-test-slave-cnb0 krb5kdc[24353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175801, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.130@KRBTEST.COM for kudu/127.13.75.190@KRBTEST.COM
I20260923 15:03:21.192716 25185 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:03:21.178861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:34207 (local address 127.13.75.130:35611)
0923 15:03:21.179092 (+ 231us) negotiation.cc:107] Waiting for socket to connect
0923 15:03:21.179135 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0923 15:03:21.179759 (+ 624us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:03:21.180044 (+ 285us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:03:21.180048 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:03:21.180292 (+ 244us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:03:21.180595 (+ 303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:03:21.180603 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:03:21.181610 (+ 1007us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:03:21.181611 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:03:21.183029 (+ 1418us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:03:21.183032 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:03:21.184328 (+ 1296us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:03:21.184329 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:03:21.184527 (+ 198us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:03:21.186947 (+ 2420us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:03:21.186959 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:03:21.188765 (+ 1806us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:03:21.190855 (+ 2090us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:03:21.190863 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:03:21.190866 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:03:21.191234 (+ 368us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:03:21.191460 (+ 226us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:03:21.191461 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:03:21.191462 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:03:21.191646 (+ 184us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:03:21.191980 (+ 334us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:03:21.191987 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:03:21.192156 (+ 169us) client_negotiation.cc:769] Sending connection context
0923 15:03:21.192271 (+ 115us) client_negotiation.cc:240] Negotiation successful
0923 15:03:21.192457 (+ 186us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":121,"threads_started":1}
I20260923 15:03:21.193192 25049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:03:21.179224 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.130:35611 (local address 127.13.75.190:34207)
0923 15:03:21.179287 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0923 15:03:21.179289 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:03:21.179305 (+ 16us) server_negotiation.cc:407] Connection header received
0923 15:03:21.179887 (+ 582us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:03:21.179888 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:03:21.179905 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:03:21.179969 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:03:21.180717 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:03:21.181484 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:03:21.183171 (+ 1687us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:03:21.183489 (+ 318us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:03:21.183540 (+ 51us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:03:21.188904 (+ 5364us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:03:21.188906 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:03:21.188906 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:03:21.188915 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:03:21.190748 (+ 1833us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:03:21.191322 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:03:21.191323 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:03:21.191323 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:03:21.191383 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:03:21.191730 (+ 347us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:03:21.191732 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:03:21.191732 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:03:21.191884 (+ 152us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:03:21.191950 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0923 15:03:21.192981 (+ 1031us) server_negotiation.cc:299] Negotiation successful
0923 15:03:21.193086 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260923 15:03:21.193810 25183 heartbeater.cc:347] Connected to a master server at 127.13.75.190:34207
I20260923 15:03:21.193910 25183 heartbeater.cc:464] Registering TS with master...
I20260923 15:03:21.194104 25183 heartbeater.cc:511] Master 127.13.75.190:34207 requested a full tablet report, sending...
I20260923 15:03:21.194502 24832 ts_manager.cc:194] Registered new tserver with Master: 8e58a506d02a4ceb9b4633414637bd44 (127.13.75.130:37027)
I20260923 15:03:21.195113 24832 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.75.130@KRBTEST.COM'} at 127.13.75.130:35611
WARNING: no policy specified for kudu/127.13.75.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.75.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.75.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.131.keytab.
Entry for principal kudu/127.13.75.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.131.keytab.
I20260923 15:03:21.218758 13614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKC4soy/build/release/bin/kudu
/tmp/dist-test-taskKC4soy/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175691491353-13614-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175691491353-13614-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175691491353-13614-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175691491353-13614-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.13.75.148:58833/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.75.131:0
--local_ip_for_outbound_sockets=127.13.75.131
--webserver_interface=127.13.75.131
--webserver_port=0
--tserver_master_addrs=127.13.75.190:34207
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.131.keytab
--principal=kudu/127.13.75.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.75.148:33553
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175691491353-13614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175691491353-13614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175691491353-13614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:03:21.331070 25188 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:03:21.331362 25188 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:03:21.331399 25188 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:03:21.333127 25188 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:03:21.333187 25188 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:03:21.333228 25188 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.75.131
I20260923 15:03:21.335377 25188 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.75.148:33553
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175691491353-13614-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175691491353-13614-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.13.75.148:58833/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.131.keytab
--principal=kudu/127.13.75.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.75.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175691491353-13614-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.75.131
--webserver_port=0
--tserver_master_addrs=127.13.75.190:34207
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25188
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.75.131
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175691491353-13614-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:03:21.335647 25188 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:03:21.335924 25188 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:03:21.343525 25196 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:03:21.343529 25193 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:03:21.344040 25194 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:03:21.344555 25188 server_base.cc:1034] running on GCE node
I20260923 15:03:21.344801 25188 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:03:21.345083 25188 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:03:21.346258 25188 hybrid_clock.cc:648] HybridClock initialized: now 1790175801346234 us; error 39 us; skew 500 ppm
Sep 23 15:03:21 dist-test-slave-cnb0 krb5kdc[24353](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175801, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:03:21.349732 25188 init.cc:377] Logged in from keytab as kudu/127.13.75.131@KRBTEST.COM (short username kudu)
I20260923 15:03:21.350281 25188 webserver.cc:460] Webserver started at http://127.13.75.131:44723/ using document root <none> and password file <none>
I20260923 15:03:21.350595 25188 fs_manager.cc:359] Metadata directory not provided
I20260923 15:03:21.350665 25188 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:03:21.350804 25188 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 23 15:03:21 dist-test-slave-cnb0 krb5kdc[24353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175801, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.131@KRBTEST.COM for HTTP/127.13.75.148@KRBTEST.COM
I20260923 15:03:21.377424 25188 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175691491353-13614-0/minicluster-data/ts-2/data/instance:
uuid: "cd4acc329928434ea82bd41579ac9e63"
format_stamp: "Formatted at 2026-09-23 15:03:21 on dist-test-slave-cnb0"
server_key: "6d9a66401cdbd538c7ba682e7d5433bb"
server_key_iv: "f49ca2dafe4e7fe6a29319c2269b001d"
server_key_version: "kuduclusterkey@0"
I20260923 15:03:21.377925 25188 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175691491353-13614-0/minicluster-data/ts-2/wal/instance:
uuid: "cd4acc329928434ea82bd41579ac9e63"
format_stamp: "Formatted at 2026-09-23 15:03:21 on dist-test-slave-cnb0"
server_key: "6d9a66401cdbd538c7ba682e7d5433bb"
server_key_iv: "f49ca2dafe4e7fe6a29319c2269b001d"
server_key_version: "kuduclusterkey@0"
I20260923 15:03:21.379870 25188 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.003s
I20260923 15:03:21.395463 25203 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:03:21.395674 25188 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:03:21.395772 25188 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175691491353-13614-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175691491353-13614-0/minicluster-data/ts-2/wal
uuid: "cd4acc329928434ea82bd41579ac9e63"
format_stamp: "Formatted at 2026-09-23 15:03:21 on dist-test-slave-cnb0"
server_key: "6d9a66401cdbd538c7ba682e7d5433bb"
server_key_iv: "f49ca2dafe4e7fe6a29319c2269b001d"
server_key_version: "kuduclusterkey@0"
I20260923 15:03:21.395881 25188 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175691491353-13614-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175691491353-13614-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175691491353-13614-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:03:21.413928 25188 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:03:21.418605 25188 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:03:21.418759 25188 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:03:21.418982 25188 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 15:03:21.419348 25188 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 15:03:21.419395 25188 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 15:03:21.419426 25188 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 15:03:21.419448 25188 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 15:03:21.425716 25188 rpc_server.cc:307] RPC server started. Bound to: 127.13.75.131:36551
I20260923 15:03:21.426156 25188 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175691491353-13614-0/minicluster-data/ts-2/data/info.pb
I20260923 15:03:21.426539 13614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKC4soy/build/release/bin/kudu as pid 25188
I20260923 15:03:21.426661 13614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175691491353-13614-0/minicluster-data/ts-2/wal/instance
I20260923 15:03:21.426934 13614 external_mini_cluster.cc:1428] Setting key 47b04c6a36f1ff12ed904204577e1991
Sep 23 15:03:21 dist-test-slave-cnb0 krb5kdc[24353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175801, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.131@KRBTEST.COM for kudu/127.13.75.190@KRBTEST.COM
I20260923 15:03:21.427194 25316 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.75.131:36551 every 8 connection(s)
I20260923 15:03:21.441823 25049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:03:21.430359 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.131:59655 (local address 127.13.75.190:34207)
0923 15:03:21.430406 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0923 15:03:21.430408 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:03:21.430766 (+ 358us) server_negotiation.cc:407] Connection header received
0923 15:03:21.431521 (+ 755us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:03:21.431522 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:03:21.431535 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:03:21.431566 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:03:21.432409 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:03:21.433228 (+ 819us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:03:21.434737 (+ 1509us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:03:21.435085 (+ 348us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:03:21.435132 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:03:21.437803 (+ 2671us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:03:21.437805 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:03:21.437806 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:03:21.437814 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:03:21.440152 (+ 2338us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:03:21.440690 (+ 538us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:03:21.440691 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:03:21.440692 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:03:21.440758 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:03:21.441054 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:03:21.441055 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:03:21.441055 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:03:21.441194 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:03:21.441235 (+ 41us) server_negotiation.cc:1035] Waiting for connection context
0923 15:03:21.441631 (+ 396us) server_negotiation.cc:299] Negotiation successful
0923 15:03:21.441721 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260923 15:03:21.442376 25319 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:03:21.430073 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:34207 (local address 127.13.75.131:59655)
0923 15:03:21.430632 (+ 559us) negotiation.cc:107] Waiting for socket to connect
0923 15:03:21.430679 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0923 15:03:21.431379 (+ 700us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:03:21.431762 (+ 383us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:03:21.431767 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:03:21.432027 (+ 260us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:03:21.432302 (+ 275us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:03:21.432308 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:03:21.433342 (+ 1034us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:03:21.433343 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:03:21.434637 (+ 1294us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:03:21.434640 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:03:21.435227 (+ 587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:03:21.435228 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:03:21.435374 (+ 146us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:03:21.436007 (+ 633us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:03:21.436017 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:03:21.437684 (+ 1667us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:03:21.440254 (+ 2570us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:03:21.440261 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:03:21.440264 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:03:21.440591 (+ 327us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:03:21.440837 (+ 246us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:03:21.440838 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:03:21.440838 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:03:21.440972 (+ 134us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:03:21.441282 (+ 310us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:03:21.441289 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:03:21.441454 (+ 165us) client_negotiation.cc:769] Sending connection context
0923 15:03:21.441967 (+ 513us) client_negotiation.cc:240] Negotiation successful
0923 15:03:21.442138 (+ 171us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":512,"thread_start_us":85,"threads_started":1}
I20260923 15:03:21.442811 25317 heartbeater.cc:347] Connected to a master server at 127.13.75.190:34207
I20260923 15:03:21.442931 25317 heartbeater.cc:464] Registering TS with master...
I20260923 15:03:21.443140 25317 heartbeater.cc:511] Master 127.13.75.190:34207 requested a full tablet report, sending...
I20260923 15:03:21.443506 24832 ts_manager.cc:194] Registered new tserver with Master: cd4acc329928434ea82bd41579ac9e63 (127.13.75.131:36551)
I20260923 15:03:21.443998 24832 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.75.131@KRBTEST.COM'} at 127.13.75.131:59655
I20260923 15:03:21.449771 13614 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260923 15:03:21.460999 25049 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:03:21.452606 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47452 (local address 127.13.75.190:34207)
0923 15:03:21.452651 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0923 15:03:21.452653 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:03:21.452992 (+ 339us) server_negotiation.cc:407] Connection header received
0923 15:03:21.453095 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:03:21.453096 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:03:21.453106 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:03:21.453135 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:03:21.453926 (+ 791us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:03:21.454670 (+ 744us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:03:21.456026 (+ 1356us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:03:21.456337 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:03:21.456384 (+ 47us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:03:21.458024 (+ 1640us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:03:21.458025 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:03:21.458026 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:03:21.458033 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:03:21.459743 (+ 1710us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:03:21.460135 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:03:21.460136 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:03:21.460136 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:03:21.460192 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:03:21.460389 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:03:21.460389 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:03:21.460390 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:03:21.460527 (+ 137us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:03:21.460604 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0923 15:03:21.460807 (+ 203us) server_negotiation.cc:299] Negotiation successful
0923 15:03:21.460885 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260923 15:03:21.463456 24832 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47452:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 15:03:21.465719 24832 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 15:03:21.474905 25331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:03:21.469786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.130:37027 (local address 127.0.0.1:33080)
0923 15:03:21.470015 (+ 229us) negotiation.cc:107] Waiting for socket to connect
0923 15:03:21.470026 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 15:03:21.470064 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:03:21.470788 (+ 724us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:03:21.470789 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:03:21.470792 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:03:21.470829 (+ 37us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:03:21.470831 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:03:21.472167 (+ 1336us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:03:21.472168 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:03:21.473569 (+ 1401us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:03:21.473571 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:03:21.474543 (+ 972us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:03:21.474544 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:03:21.474630 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:03:21.474650 (+ 20us) client_negotiation.cc:769] Sending connection context
0923 15:03:21.474697 (+ 47us) client_negotiation.cc:240] Negotiation successful
0923 15:03:21.474754 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":212,"mutex_wait_us":51,"thread_start_us":105,"threads_started":1}
I20260923 15:03:21.475211 25332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:03:21.469981 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59044 (local address 127.13.75.129:43021)
0923 15:03:21.470161 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0923 15:03:21.470166 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 15:03:21.470176 (+ 10us) server_negotiation.cc:407] Connection header received
0923 15:03:21.470215 (+ 39us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:03:21.470217 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:03:21.470234 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:03:21.470824 (+ 590us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:03:21.470890 (+ 66us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:03:21.472068 (+ 1178us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:03:21.473880 (+ 1812us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:03:21.474591 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:03:21.474949 (+ 358us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:03:21.475038 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0923 15:03:21.475065 (+ 27us) server_negotiation.cc:299] Negotiation successful
0923 15:03:21.475118 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":68,"threads_started":1}
I20260923 15:03:21.474913 25329 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:03:21.469665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.129:43021 (local address 127.0.0.1:59044)
0923 15:03:21.469797 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0923 15:03:21.469837 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0923 15:03:21.469899 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:03:21.470364 (+ 465us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:03:21.470367 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:03:21.470370 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:03:21.470447 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:03:21.470449 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:03:21.472169 (+ 1720us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:03:21.472169 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:03:21.473800 (+ 1631us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:03:21.473802 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:03:21.474674 (+ 872us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:03:21.474675 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:03:21.474727 (+ 52us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:03:21.474729 (+ 2us) client_negotiation.cc:769] Sending connection context
0923 15:03:21.474757 (+ 28us) client_negotiation.cc:240] Negotiation successful
0923 15:03:21.474783 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"mutex_wait_us":338,"thread_start_us":78,"threads_started":1}
I20260923 15:03:21.476236 25334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:03:21.470416 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33080 (local address 127.13.75.130:37027)
0923 15:03:21.470618 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0923 15:03:21.470623 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 15:03:21.470633 (+ 10us) server_negotiation.cc:407] Connection header received
0923 15:03:21.470671 (+ 38us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:03:21.470673 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:03:21.470688 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:03:21.470925 (+ 237us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:03:21.470996 (+ 71us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:03:21.472055 (+ 1059us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:03:21.473669 (+ 1614us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:03:21.474429 (+ 760us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:03:21.476004 (+ 1575us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:03:21.476094 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0923 15:03:21.476119 (+ 25us) server_negotiation.cc:299] Negotiation successful
0923 15:03:21.476157 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":163,"thread_start_us":101,"threads_started":1}
I20260923 15:03:21.476408 24979 tablet_service.cc:1467] Processing CreateTablet for tablet cab05622a3d6462ababdeea70938b570 (DEFAULT_TABLE table=test-table [id=32cebb286a194c72963e9c8f208e6646]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 15:03:21.476871 24979 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cab05622a3d6462ababdeea70938b570. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:03:21.476915 25098 tablet_service.cc:1467] Processing CreateTablet for tablet cab05622a3d6462ababdeea70938b570 (DEFAULT_TABLE table=test-table [id=32cebb286a194c72963e9c8f208e6646]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 15:03:21.477296 25098 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cab05622a3d6462ababdeea70938b570. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:03:21.479512 25331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:03:21.469890 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.131:36551 (local address 127.0.0.1:49144)
0923 15:03:21.475008 (+ 5118us) negotiation.cc:107] Waiting for socket to connect
0923 15:03:21.475019 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 15:03:21.475062 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:03:21.475491 (+ 429us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:03:21.475491 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:03:21.475493 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:03:21.475524 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:03:21.475525 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:03:21.477227 (+ 1702us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:03:21.477228 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:03:21.478237 (+ 1009us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:03:21.478240 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:03:21.479293 (+ 1053us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:03:21.479294 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:03:21.479356 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:03:21.479360 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 15:03:21.479387 (+ 27us) client_negotiation.cc:240] Negotiation successful
0923 15:03:21.479426 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":5106,"thread_start_us":71,"threads_started":1}
I20260923 15:03:21.480335 25333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:03:21.470105 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49144 (local address 127.13.75.131:36551)
0923 15:03:21.470353 (+ 248us) server_negotiation.cc:206] Beginning negotiation
0923 15:03:21.470357 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 15:03:21.475316 (+ 4959us) server_negotiation.cc:407] Connection header received
0923 15:03:21.475366 (+ 50us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:03:21.475369 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:03:21.475391 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:03:21.475466 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:03:21.475764 (+ 298us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:03:21.477063 (+ 1299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:03:21.478410 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:03:21.479199 (+ 789us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:03:21.479817 (+ 618us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:03:21.479903 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0923 15:03:21.479931 (+ 28us) server_negotiation.cc:299] Negotiation successful
0923 15:03:21.480258 (+ 327us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":216,"thread_start_us":75,"threads_started":1}
I20260923 15:03:21.481172 25336 tablet_bootstrap.cc:492] T cab05622a3d6462ababdeea70938b570 P 8e58a506d02a4ceb9b4633414637bd44: Bootstrap starting.
I20260923 15:03:21.481161 25251 tablet_service.cc:1467] Processing CreateTablet for tablet cab05622a3d6462ababdeea70938b570 (DEFAULT_TABLE table=test-table [id=32cebb286a194c72963e9c8f208e6646]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 15:03:21.481493 25251 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet cab05622a3d6462ababdeea70938b570. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:03:21.482795 25336 tablet_bootstrap.cc:654] T cab05622a3d6462ababdeea70938b570 P 8e58a506d02a4ceb9b4633414637bd44: Neither blocks nor log segments found. Creating new log.
I20260923 15:03:21.483218 25336 log.cc:824] T cab05622a3d6462ababdeea70938b570 P 8e58a506d02a4ceb9b4633414637bd44: Log is configured to *not* fsync() on all Append() calls
I20260923 15:03:21.483614 25335 tablet_bootstrap.cc:492] T cab05622a3d6462ababdeea70938b570 P 73ae947088af44479a73b84f225c691b: Bootstrap starting.
I20260923 15:03:21.484388 25335 tablet_bootstrap.cc:654] T cab05622a3d6462ababdeea70938b570 P 73ae947088af44479a73b84f225c691b: Neither blocks nor log segments found. Creating new log.
I20260923 15:03:21.484715 25335 log.cc:824] T cab05622a3d6462ababdeea70938b570 P 73ae947088af44479a73b84f225c691b: Log is configured to *not* fsync() on all Append() calls
I20260923 15:03:21.485078 25338 tablet_bootstrap.cc:492] T cab05622a3d6462ababdeea70938b570 P cd4acc329928434ea82bd41579ac9e63: Bootstrap starting.
I20260923 15:03:21.486934 25336 tablet_bootstrap.cc:492] T cab05622a3d6462ababdeea70938b570 P 8e58a506d02a4ceb9b4633414637bd44: No bootstrap required, opened a new log
I20260923 15:03:21.487025 25336 ts_tablet_manager.cc:1397] T cab05622a3d6462ababdeea70938b570 P 8e58a506d02a4ceb9b4633414637bd44: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20260923 15:03:21.489372 25336 raft_consensus.cc:348] T cab05622a3d6462ababdeea70938b570 P 8e58a506d02a4ceb9b4633414637bd44 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73ae947088af44479a73b84f225c691b" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 43021 } } peers { permanent_uuid: "cd4acc329928434ea82bd41579ac9e63" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 36551 } } peers { permanent_uuid: "8e58a506d02a4ceb9b4633414637bd44" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 37027 } }
I20260923 15:03:21.489578 25336 raft_consensus.cc:374] T cab05622a3d6462ababdeea70938b570 P 8e58a506d02a4ceb9b4633414637bd44 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:03:21.489616 25336 raft_consensus.cc:729] T cab05622a3d6462ababdeea70938b570 P 8e58a506d02a4ceb9b4633414637bd44 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 8e58a506d02a4ceb9b4633414637bd44, State: Initialized, Role: FOLLOWER
I20260923 15:03:21.489795 25336 consensus_queue.cc:260] T cab05622a3d6462ababdeea70938b570 P 8e58a506d02a4ceb9b4633414637bd44 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73ae947088af44479a73b84f225c691b" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 43021 } } peers { permanent_uuid: "cd4acc329928434ea82bd41579ac9e63" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 36551 } } peers { permanent_uuid: "8e58a506d02a4ceb9b4633414637bd44" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 37027 } }
I20260923 15:03:21.490109 25336 ts_tablet_manager.cc:1428] T cab05622a3d6462ababdeea70938b570 P 8e58a506d02a4ceb9b4633414637bd44: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260923 15:03:21.486126 25338 tablet_bootstrap.cc:654] T cab05622a3d6462ababdeea70938b570 P cd4acc329928434ea82bd41579ac9e63: Neither blocks nor log segments found. Creating new log.
I20260923 15:03:21.490278 25183 heartbeater.cc:503] Master 127.13.75.190:34207 was elected leader, sending a full tablet report...
I20260923 15:03:21.490638 25338 log.cc:824] T cab05622a3d6462ababdeea70938b570 P cd4acc329928434ea82bd41579ac9e63: Log is configured to *not* fsync() on all Append() calls
I20260923 15:03:21.491665 25335 tablet_bootstrap.cc:492] T cab05622a3d6462ababdeea70938b570 P 73ae947088af44479a73b84f225c691b: No bootstrap required, opened a new log
I20260923 15:03:21.491750 25335 ts_tablet_manager.cc:1397] T cab05622a3d6462ababdeea70938b570 P 73ae947088af44479a73b84f225c691b: Time spent bootstrapping tablet: real 0.008s user 0.001s sys 0.000s
I20260923 15:03:21.494683 25335 raft_consensus.cc:348] T cab05622a3d6462ababdeea70938b570 P 73ae947088af44479a73b84f225c691b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73ae947088af44479a73b84f225c691b" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 43021 } } peers { permanent_uuid: "cd4acc329928434ea82bd41579ac9e63" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 36551 } } peers { permanent_uuid: "8e58a506d02a4ceb9b4633414637bd44" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 37027 } }
I20260923 15:03:21.494922 25335 raft_consensus.cc:374] T cab05622a3d6462ababdeea70938b570 P 73ae947088af44479a73b84f225c691b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:03:21.495028 25335 raft_consensus.cc:729] T cab05622a3d6462ababdeea70938b570 P 73ae947088af44479a73b84f225c691b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 73ae947088af44479a73b84f225c691b, State: Initialized, Role: FOLLOWER
I20260923 15:03:21.495189 25335 consensus_queue.cc:260] T cab05622a3d6462ababdeea70938b570 P 73ae947088af44479a73b84f225c691b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73ae947088af44479a73b84f225c691b" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 43021 } } peers { permanent_uuid: "cd4acc329928434ea82bd41579ac9e63" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 36551 } } peers { permanent_uuid: "8e58a506d02a4ceb9b4633414637bd44" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 37027 } }
I20260923 15:03:21.495513 25335 ts_tablet_manager.cc:1428] T cab05622a3d6462ababdeea70938b570 P 73ae947088af44479a73b84f225c691b: Time spent starting tablet: real 0.004s user 0.000s sys 0.004s
I20260923 15:03:21.495743 25045 heartbeater.cc:503] Master 127.13.75.190:34207 was elected leader, sending a full tablet report...
I20260923 15:03:21.496083 25338 tablet_bootstrap.cc:492] T cab05622a3d6462ababdeea70938b570 P cd4acc329928434ea82bd41579ac9e63: No bootstrap required, opened a new log
I20260923 15:03:21.496160 25338 ts_tablet_manager.cc:1397] T cab05622a3d6462ababdeea70938b570 P cd4acc329928434ea82bd41579ac9e63: Time spent bootstrapping tablet: real 0.011s user 0.002s sys 0.000s
I20260923 15:03:21.498505 25338 raft_consensus.cc:348] T cab05622a3d6462ababdeea70938b570 P cd4acc329928434ea82bd41579ac9e63 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73ae947088af44479a73b84f225c691b" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 43021 } } peers { permanent_uuid: "cd4acc329928434ea82bd41579ac9e63" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 36551 } } peers { permanent_uuid: "8e58a506d02a4ceb9b4633414637bd44" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 37027 } }
I20260923 15:03:21.498699 25338 raft_consensus.cc:374] T cab05622a3d6462ababdeea70938b570 P cd4acc329928434ea82bd41579ac9e63 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:03:21.498739 25338 raft_consensus.cc:729] T cab05622a3d6462ababdeea70938b570 P cd4acc329928434ea82bd41579ac9e63 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cd4acc329928434ea82bd41579ac9e63, State: Initialized, Role: FOLLOWER
I20260923 15:03:21.498970 25338 consensus_queue.cc:260] T cab05622a3d6462ababdeea70938b570 P cd4acc329928434ea82bd41579ac9e63 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73ae947088af44479a73b84f225c691b" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 43021 } } peers { permanent_uuid: "cd4acc329928434ea82bd41579ac9e63" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 36551 } } peers { permanent_uuid: "8e58a506d02a4ceb9b4633414637bd44" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 37027 } }
I20260923 15:03:21.499317 25338 ts_tablet_manager.cc:1428] T cab05622a3d6462ababdeea70938b570 P cd4acc329928434ea82bd41579ac9e63: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260923 15:03:21.499984 25317 heartbeater.cc:503] Master 127.13.75.190:34207 was elected leader, sending a full tablet report...
Sep 23 15:03:21 dist-test-slave-cnb0 krb5kdc[24353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175801, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.190@KRBTEST.COM for HTTP/127.13.75.148@KRBTEST.COM
W20260923 15:03:21.661641 25046 tablet.cc:2369] T cab05622a3d6462ababdeea70938b570 P 73ae947088af44479a73b84f225c691b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260923 15:03:21.678604 25184 tablet.cc:2369] T cab05622a3d6462ababdeea70938b570 P 8e58a506d02a4ceb9b4633414637bd44: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260923 15:03:21.680758 25318 tablet.cc:2369] T cab05622a3d6462ababdeea70938b570 P cd4acc329928434ea82bd41579ac9e63: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 15:03:21.723680 25342 raft_consensus.cc:482] T cab05622a3d6462ababdeea70938b570 P 73ae947088af44479a73b84f225c691b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 15:03:21.723857 25342 raft_consensus.cc:504] T cab05622a3d6462ababdeea70938b570 P 73ae947088af44479a73b84f225c691b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73ae947088af44479a73b84f225c691b" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 43021 } } peers { permanent_uuid: "cd4acc329928434ea82bd41579ac9e63" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 36551 } } peers { permanent_uuid: "8e58a506d02a4ceb9b4633414637bd44" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 37027 } }
I20260923 15:03:21.724304 25342 leader_election.cc:288] T cab05622a3d6462ababdeea70938b570 P 73ae947088af44479a73b84f225c691b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers cd4acc329928434ea82bd41579ac9e63 (127.13.75.131:36551), 8e58a506d02a4ceb9b4633414637bd44 (127.13.75.130:37027)
I20260923 15:03:21.728859 25333 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:03:21.724955 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.129:58665 (local address 127.13.75.131:36551)
0923 15:03:21.725007 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0923 15:03:21.725009 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:03:21.725021 (+ 12us) server_negotiation.cc:407] Connection header received
0923 15:03:21.725056 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:03:21.725057 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:03:21.725069 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:03:21.725143 (+ 74us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:03:21.725404 (+ 261us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:03:21.726225 (+ 821us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:03:21.727875 (+ 1650us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:03:21.728424 (+ 549us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:03:21.728479 (+ 55us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:03:21.728488 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0923 15:03:21.728724 (+ 236us) server_negotiation.cc:299] Negotiation successful
0923 15:03:21.728767 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260923 15:03:21.728991 25334 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:03:21.724760 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.129:47757 (local address 127.13.75.130:37027)
0923 15:03:21.724805 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0923 15:03:21.724808 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:03:21.725273 (+ 465us) server_negotiation.cc:407] Connection header received
0923 15:03:21.725373 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:03:21.725374 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:03:21.725387 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:03:21.725416 (+ 29us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:03:21.725634 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:03:21.726310 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:03:21.728040 (+ 1730us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:03:21.728588 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:03:21.728642 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:03:21.728650 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0923 15:03:21.728874 (+ 224us) server_negotiation.cc:299] Negotiation successful
0923 15:03:21.728923 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260923 15:03:21.729230 25345 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:03:21.724528 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.130:37027 (local address 127.13.75.129:47757)
0923 15:03:21.725214 (+ 686us) negotiation.cc:107] Waiting for socket to connect
0923 15:03:21.725224 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 15:03:21.725299 (+ 75us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:03:21.725484 (+ 185us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:03:21.725485 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:03:21.725486 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:03:21.725565 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:03:21.725566 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:03:21.726792 (+ 1226us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:03:21.726793 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:03:21.727927 (+ 1134us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:03:21.727930 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:03:21.728710 (+ 780us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:03:21.728711 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:03:21.728793 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:03:21.728796 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 15:03:21.729102 (+ 306us) client_negotiation.cc:240] Negotiation successful
0923 15:03:21.729146 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":659,"mutex_wait_us":13,"thread_start_us":105,"threads_started":1}
I20260923 15:03:21.729557 25124 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cab05622a3d6462ababdeea70938b570" candidate_uuid: "73ae947088af44479a73b84f225c691b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e58a506d02a4ceb9b4633414637bd44" is_pre_election: true
I20260923 15:03:21.729743 25124 raft_consensus.cc:2454] T cab05622a3d6462ababdeea70938b570 P 8e58a506d02a4ceb9b4633414637bd44 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 73ae947088af44479a73b84f225c691b in term 0.
I20260923 15:03:21.730048 24934 leader_election.cc:302] T cab05622a3d6462ababdeea70938b570 P 73ae947088af44479a73b84f225c691b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 73ae947088af44479a73b84f225c691b, 8e58a506d02a4ceb9b4633414637bd44; no voters:
I20260923 15:03:21.730088 25344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:03:21.724524 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.131:36551 (local address 127.13.75.129:58665)
0923 15:03:21.724875 (+ 351us) negotiation.cc:107] Waiting for socket to connect
0923 15:03:21.724899 (+ 24us) client_negotiation.cc:174] Beginning negotiation
0923 15:03:21.724961 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:03:21.725206 (+ 245us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:03:21.725207 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:03:21.725210 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:03:21.725311 (+ 101us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:03:21.725314 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:03:21.726412 (+ 1098us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:03:21.726413 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:03:21.727738 (+ 1325us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:03:21.727742 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:03:21.728555 (+ 813us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:03:21.728556 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:03:21.728630 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:03:21.728635 (+ 5us) client_negotiation.cc:769] Sending connection context
0923 15:03:21.729405 (+ 770us) client_negotiation.cc:240] Negotiation successful
0923 15:03:21.729983 (+ 578us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":331,"thread_start_us":109,"threads_started":1}
I20260923 15:03:21.730224 25342 raft_consensus.cc:2781] T cab05622a3d6462ababdeea70938b570 P 73ae947088af44479a73b84f225c691b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 15:03:21.730276 25342 raft_consensus.cc:482] T cab05622a3d6462ababdeea70938b570 P 73ae947088af44479a73b84f225c691b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 15:03:21.730296 25342 raft_consensus.cc:3037] T cab05622a3d6462ababdeea70938b570 P 73ae947088af44479a73b84f225c691b [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:03:21.730440 25269 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cab05622a3d6462ababdeea70938b570" candidate_uuid: "73ae947088af44479a73b84f225c691b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd4acc329928434ea82bd41579ac9e63" is_pre_election: true
I20260923 15:03:21.730607 25269 raft_consensus.cc:2454] T cab05622a3d6462ababdeea70938b570 P cd4acc329928434ea82bd41579ac9e63 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 73ae947088af44479a73b84f225c691b in term 0.
I20260923 15:03:21.731300 25342 raft_consensus.cc:504] T cab05622a3d6462ababdeea70938b570 P 73ae947088af44479a73b84f225c691b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73ae947088af44479a73b84f225c691b" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 43021 } } peers { permanent_uuid: "cd4acc329928434ea82bd41579ac9e63" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 36551 } } peers { permanent_uuid: "8e58a506d02a4ceb9b4633414637bd44" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 37027 } }
I20260923 15:03:21.731452 25342 leader_election.cc:288] T cab05622a3d6462ababdeea70938b570 P 73ae947088af44479a73b84f225c691b [CANDIDATE]: Term 1 election: Requested vote from peers cd4acc329928434ea82bd41579ac9e63 (127.13.75.131:36551), 8e58a506d02a4ceb9b4633414637bd44 (127.13.75.130:37027)
I20260923 15:03:21.731655 25124 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cab05622a3d6462ababdeea70938b570" candidate_uuid: "73ae947088af44479a73b84f225c691b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "8e58a506d02a4ceb9b4633414637bd44"
I20260923 15:03:21.731701 25269 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "cab05622a3d6462ababdeea70938b570" candidate_uuid: "73ae947088af44479a73b84f225c691b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "cd4acc329928434ea82bd41579ac9e63"
I20260923 15:03:21.731735 25124 raft_consensus.cc:3037] T cab05622a3d6462ababdeea70938b570 P 8e58a506d02a4ceb9b4633414637bd44 [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:03:21.731798 25269 raft_consensus.cc:3037] T cab05622a3d6462ababdeea70938b570 P cd4acc329928434ea82bd41579ac9e63 [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:03:21.732623 25124 raft_consensus.cc:2454] T cab05622a3d6462ababdeea70938b570 P 8e58a506d02a4ceb9b4633414637bd44 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 73ae947088af44479a73b84f225c691b in term 1.
I20260923 15:03:21.732782 25269 raft_consensus.cc:2454] T cab05622a3d6462ababdeea70938b570 P cd4acc329928434ea82bd41579ac9e63 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 73ae947088af44479a73b84f225c691b in term 1.
I20260923 15:03:21.732797 24934 leader_election.cc:302] T cab05622a3d6462ababdeea70938b570 P 73ae947088af44479a73b84f225c691b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 73ae947088af44479a73b84f225c691b, 8e58a506d02a4ceb9b4633414637bd44; no voters:
I20260923 15:03:21.732877 25342 raft_consensus.cc:2781] T cab05622a3d6462ababdeea70938b570 P 73ae947088af44479a73b84f225c691b [term 1 FOLLOWER]: Leader election won for term 1
I20260923 15:03:21.733013 25342 raft_consensus.cc:686] T cab05622a3d6462ababdeea70938b570 P 73ae947088af44479a73b84f225c691b [term 1 LEADER]: Becoming Leader. State: Replica: 73ae947088af44479a73b84f225c691b, State: Running, Role: LEADER
I20260923 15:03:21.733103 25342 consensus_queue.cc:237] T cab05622a3d6462ababdeea70938b570 P 73ae947088af44479a73b84f225c691b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73ae947088af44479a73b84f225c691b" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 43021 } } peers { permanent_uuid: "cd4acc329928434ea82bd41579ac9e63" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 36551 } } peers { permanent_uuid: "8e58a506d02a4ceb9b4633414637bd44" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 37027 } }
I20260923 15:03:21.734066 24832 catalog_manager.cc:5495] T cab05622a3d6462ababdeea70938b570 P 73ae947088af44479a73b84f225c691b reported cstate change: term changed from 0 to 1, leader changed from <none> to 73ae947088af44479a73b84f225c691b (127.13.75.129). New cstate: current_term: 1 leader_uuid: "73ae947088af44479a73b84f225c691b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "73ae947088af44479a73b84f225c691b" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 43021 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "cd4acc329928434ea82bd41579ac9e63" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 36551 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "8e58a506d02a4ceb9b4633414637bd44" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 37027 } health_report { overall_health: UNKNOWN } } }
I20260923 15:03:21.758458 25332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:03:21.754930 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59060 (local address 127.13.75.129:43021)
0923 15:03:21.754978 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0923 15:03:21.754981 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:03:21.755032 (+ 51us) server_negotiation.cc:407] Connection header received
0923 15:03:21.755120 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:03:21.755121 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:03:21.755148 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:03:21.755216 (+ 68us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 15:03:21.755413 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:03:21.756175 (+ 762us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:03:21.757277 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:03:21.757574 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:03:21.757642 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:03:21.757855 (+ 213us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 15:03:21.757989 (+ 134us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 15:03:21.758070 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0923 15:03:21.758317 (+ 247us) server_negotiation.cc:299] Negotiation successful
0923 15:03:21.758346 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260923 15:03:21.760898 25269 raft_consensus.cc:1261] T cab05622a3d6462ababdeea70938b570 P cd4acc329928434ea82bd41579ac9e63 [term 1 FOLLOWER]: Refusing update from remote peer 73ae947088af44479a73b84f225c691b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 15:03:21.761273 25124 raft_consensus.cc:1261] T cab05622a3d6462ababdeea70938b570 P 8e58a506d02a4ceb9b4633414637bd44 [term 1 FOLLOWER]: Refusing update from remote peer 73ae947088af44479a73b84f225c691b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 15:03:21.761261 25342 consensus_queue.cc:1035] T cab05622a3d6462ababdeea70938b570 P 73ae947088af44479a73b84f225c691b [LEADER]: Connected to new peer: Peer: permanent_uuid: "cd4acc329928434ea82bd41579ac9e63" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 36551 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 15:03:21.761525 25342 consensus_queue.cc:1035] T cab05622a3d6462ababdeea70938b570 P 73ae947088af44479a73b84f225c691b [LEADER]: Connected to new peer: Peer: permanent_uuid: "8e58a506d02a4ceb9b4633414637bd44" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 37027 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 15:03:21.766539 25350 mvcc.cc:205] Tried to move back new op lower bound from 7332560084009336832 to 7332560083899301888. Current Snapshot: MvccSnapshot[applied={T|T < 7332560084009336832}]
I20260923 15:03:21.768234 25353 mvcc.cc:205] Tried to move back new op lower bound from 7332560084009336832 to 7332560083899301888. Current Snapshot: MvccSnapshot[applied={T|T < 7332560084009336832 or (T in {7332560084009336832})}]
I20260923 15:03:21.769124 25351 mvcc.cc:205] Tried to move back new op lower bound from 7332560084009336832 to 7332560083899301888. Current Snapshot: MvccSnapshot[applied={T|T < 7332560084009336832}]
I20260923 15:03:21.772215 24832 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47452:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 15:03:21.772330 24832 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:47452:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 15:03:21.773072 24832 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 7d6d822f805249e899cd6dd8ba134b0c: Sending DeleteTablet for 3 replicas of tablet cab05622a3d6462ababdeea70938b570
I20260923 15:03:21.773412 25098 tablet_service.cc:1514] Processing DeleteTablet for tablet cab05622a3d6462ababdeea70938b570 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 15:03:21 UTC) from {username='kudu', principal='kudu/127.13.75.190@KRBTEST.COM'} at 127.0.0.1:33080
I20260923 15:03:21.773885 25251 tablet_service.cc:1514] Processing DeleteTablet for tablet cab05622a3d6462ababdeea70938b570 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 15:03:21 UTC) from {username='kudu', principal='kudu/127.13.75.190@KRBTEST.COM'} at 127.0.0.1:49144
I20260923 15:03:21.774199 24979 tablet_service.cc:1514] Processing DeleteTablet for tablet cab05622a3d6462ababdeea70938b570 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 15:03:21 UTC) from {username='kudu', principal='kudu/127.13.75.190@KRBTEST.COM'} at 127.0.0.1:59044
I20260923 15:03:21.774503 13614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKC4soy/build/release/bin/kudu with pid 24913
Sep 23 15:03:21 dist-test-slave-cnb0 krb5kdc[24353](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175801, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.190@KRBTEST.COM for HTTP/127.13.75.148@KRBTEST.COM
I20260923 15:03:21.785458 25361 tablet_replica.cc:330] T cab05622a3d6462ababdeea70938b570 P cd4acc329928434ea82bd41579ac9e63: stopping tablet replica
I20260923 15:03:21.785648 25361 raft_consensus.cc:2229] T cab05622a3d6462ababdeea70938b570 P cd4acc329928434ea82bd41579ac9e63 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 15:03:21.785815 25361 raft_consensus.cc:2258] T cab05622a3d6462ababdeea70938b570 P cd4acc329928434ea82bd41579ac9e63 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 15:03:21.786458 25361 ts_tablet_manager.cc:1905] T cab05622a3d6462ababdeea70938b570 P cd4acc329928434ea82bd41579ac9e63: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 15:03:21.788460 25361 ts_tablet_manager.cc:1918] T cab05622a3d6462ababdeea70938b570 P cd4acc329928434ea82bd41579ac9e63: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 15:03:21.788561 25361 log.cc:1196] T cab05622a3d6462ababdeea70938b570 P cd4acc329928434ea82bd41579ac9e63: Deleting WAL directory at /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175691491353-13614-0/minicluster-data/ts-2/wal/wals/cab05622a3d6462ababdeea70938b570
I20260923 15:03:21.789292 25361 ts_tablet_manager.cc:1939] T cab05622a3d6462ababdeea70938b570 P cd4acc329928434ea82bd41579ac9e63: Deleting consensus metadata
I20260923 15:03:21.789860 24818 catalog_manager.cc:4841] TS cd4acc329928434ea82bd41579ac9e63 (127.13.75.131:36551): tablet cab05622a3d6462ababdeea70938b570 (table test-table [id=32cebb286a194c72963e9c8f208e6646]) successfully deleted
I20260923 15:03:21.793224 25360 tablet_replica.cc:330] T cab05622a3d6462ababdeea70938b570 P 8e58a506d02a4ceb9b4633414637bd44: stopping tablet replica
I20260923 15:03:21.793385 25360 raft_consensus.cc:2229] T cab05622a3d6462ababdeea70938b570 P 8e58a506d02a4ceb9b4633414637bd44 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 15:03:21.793531 25360 raft_consensus.cc:2258] T cab05622a3d6462ababdeea70938b570 P 8e58a506d02a4ceb9b4633414637bd44 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 15:03:21.794234 25360 ts_tablet_manager.cc:1905] T cab05622a3d6462ababdeea70938b570 P 8e58a506d02a4ceb9b4633414637bd44: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 15:03:21.796030 25360 ts_tablet_manager.cc:1918] T cab05622a3d6462ababdeea70938b570 P 8e58a506d02a4ceb9b4633414637bd44: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 15:03:21.796123 25360 log.cc:1196] T cab05622a3d6462ababdeea70938b570 P 8e58a506d02a4ceb9b4633414637bd44: Deleting WAL directory at /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegration.1790175691491353-13614-0/minicluster-data/ts-1/wal/wals/cab05622a3d6462ababdeea70938b570
I20260923 15:03:21.796418 25360 ts_tablet_manager.cc:1939] T cab05622a3d6462ababdeea70938b570 P 8e58a506d02a4ceb9b4633414637bd44: Deleting consensus metadata
I20260923 15:03:21.796954 24819 catalog_manager.cc:4841] TS 8e58a506d02a4ceb9b4633414637bd44 (127.13.75.130:37027): tablet cab05622a3d6462ababdeea70938b570 (table test-table [id=32cebb286a194c72963e9c8f208e6646]) successfully deleted
W20260923 15:03:21.801050 24818 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.75.129:43021 (error 108)
I20260923 15:03:21.801358 13614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKC4soy/build/release/bin/kudu with pid 25053
I20260923 15:03:21.810030 13614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKC4soy/build/release/bin/kudu with pid 25188
I20260923 15:03:21.810705 25331 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:03:21.810135 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.129:43021 (local address 127.0.0.1:59062)
0923 15:03:21.810587 (+ 452us) negotiation.cc:107] Waiting for socket to connect
0923 15:03:21.810657 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.129:43021: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":424}
W20260923 15:03:21.810998 24818 catalog_manager.cc:4584] TS 73ae947088af44479a73b84f225c691b (127.13.75.129:43021): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet cab05622a3d6462ababdeea70938b570: Network error: Client connection negotiation failed: client connection to 127.13.75.129:43021: connect: Connection refused (error 111)
I20260923 15:03:21.817461 13614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKC4soy/build/release/bin/kudu with pid 24800
I20260923 15:03:21.824235 13614 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260923 15:03:22.168880 13614 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260923 15:03:22.169008 13614 mini_ranger.cc:66] Stopping Ranger...
I20260923 15:03:22.514150 13614 mini_ranger.cc:68] Stopped Ranger
2026-09-23 15:03:22.514 UTC [24383] LOG: received smart shutdown request
2026-09-23 15:03:22.518 UTC [24383] LOG: background worker "logical replication launcher" (PID 24392) exited with exit code 1
2026-09-23 15:03:22.519 UTC [24387] LOG: shutting down
2026-09-23 15:03:22.715 UTC [24383] LOG: database system is shut down
2026-09-23T15:03:22Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegration (52288 ms)
[ RUN ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers
Loading random data
Initializing database '/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 15:03:22 dist-test-slave-cnb0 krb5kdc[25373](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 15:03:22 dist-test-slave-cnb0 krb5kdc[25373](info): set up 2 sockets
Sep 23 15:03:22 dist-test-slave-cnb0 krb5kdc[25373](info): commencing operation
krb5kdc: starting...
W20260923 15:03:23.887882 13614 mini_kdc.cc:121] Time spent starting KDC: real 1.049s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/krb5kdc/test-user.keytab.
Sep 23 15:03:23 dist-test-slave-cnb0 krb5kdc[25373](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175803, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T15:03:23Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T15:03:23Z Disabled control of system clock
WARNING: no policy specified for rangeradmin/127.13.75.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangeradmin/127.13.75.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangeradmin/127.13.75.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/krb5kdc/rangeradmin_127.13.75.148@KRBTEST.COM.keytab.
Entry for principal rangeradmin/127.13.75.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/krb5kdc/rangeradmin_127.13.75.148@KRBTEST.COM.keytab.
WARNING: no policy specified for rangerlookup/127.13.75.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerlookup/127.13.75.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerlookup/127.13.75.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/krb5kdc/rangerlookup_127.13.75.148@KRBTEST.COM.keytab.
Entry for principal rangerlookup/127.13.75.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/krb5kdc/rangerlookup_127.13.75.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.13.75.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "HTTP/127.13.75.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.13.75.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/krb5kdc/HTTP_127.13.75.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.13.75.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/krb5kdc/HTTP_127.13.75.148@KRBTEST.COM.keytab.
I20260923 15:03:24.057838 13614 mini_postgres.cc:62] Running initdb...
The files belonging to this database system will be owned by user "slave".
This user must also own the server process.
The database cluster will be initialized with locale "C".
The default database encoding has accordingly been set to "SQL_ASCII".
The default text search configuration will be set to "english".
Data page checksums are disabled.
creating directory /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/minicluster-data/postgres ... ok
creating subdirectories ... ok
selecting dynamic shared memory implementation ... posix
selecting default max_connections ... 100
selecting default shared_buffers ... 128MB
selecting default time zone ... Etc/UTC
creating configuration files ... ok
running bootstrap script ... ok
performing post-bootstrap initialization ... ok
syncing data to disk ... ok
initdb: warning: enabling "trust" authentication for local connections
You can change this by editing pg_hba.conf or using the option -A, or
--auth-local and --auth-host, the next time you run initdb.
Success. You can now start the database server using:
/tmp/dist-test-taskKC4soy/build/release/bin/postgres/pg_ctl -D /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/minicluster-data/postgres -l logfile start
2026-09-23 15:03:26.647 UTC [25403] LOG: starting PostgreSQL 12.2 on x86_64-pc-linux-gnu, compiled by cc (GCC) 8.3.1 20190311 (Red Hat 8.3.1-3), 64-bit
2026-09-23 15:03:26.647 UTC [25403] LOG: listening on IPv4 address "127.13.75.148", port 46271
2026-09-23 15:03:26.651 UTC [25403] LOG: listening on Unix socket "/tmp/.s.PGSQL.46271"
2026-09-23 15:03:26.681 UTC [25406] LOG: database system was shut down at 2026-09-23 15:03:25 UTC
2026-09-23 15:03:26.685 UTC [25403] LOG: database system is ready to accept connections
I20260923 15:03:27.647732 13614 mini_postgres.cc:96] Postgres bound to 46271
2026-09-23 15:03:27.653 UTC [25414] FATAL: database "slave" does not exist
127.13.75.148:46271 - accepting connections
I20260923 15:03:27.654526 13614 mini_ranger.cc:161] Starting Ranger...
I20260923 15:03:27.673753 13614 mini_ranger.cc:84] Created miniranger Postgres user
I20260923 15:03:28.214148 13614 mini_ranger.cc:87] Created ranger Postgres database
I20260923 15:03:28.214221 13614 mini_ranger.cc:178] Starting Ranger out of /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/minicluster-data/ranger-admin
2026-09-23 15:03:28,494 [I] DB FLAVOR :POSTGRES
2026-09-23 15:03:28,495 [I] --------- Verifying Ranger DB connection ---------
2026-09-23 15:03:28,495 [I] Checking connection..
2026-09-23 15:03:28,495 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKC4soy/build/release/bin/postgresql.jar:/tmp/dist-test-taskKC4soy/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.75.148:46271/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select 1;"
2026-09-23 15:03:28,761 [I] Checking connection passed.
2026-09-23 15:03:28,761 [I] --------- Verifying version history table ---------
2026-09-23 15:03:28,761 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKC4soy/build/release/bin/postgresql.jar:/tmp/dist-test-taskKC4soy/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.75.148:46271/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-23 15:03:29,030 [I] Table x_db_version_h does not exist in database ranger
2026-09-23 15:03:29,031 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKC4soy/build/release/bin/postgresql.jar:/tmp/dist-test-taskKC4soy/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.75.148:46271/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-23 15:03:29,301 [I] Table x_db_version_h does not exist in database ranger
2026-09-23 15:03:29,301 [I] Importing x_db_version_h table schema to database ranger from file: create_dbversion_catalog.sql
2026-09-23 15:03:29,301 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKC4soy/build/release/bin/postgresql.jar:/tmp/dist-test-taskKC4soy/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.75.148:46271/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskKC4soy/thirdparty/src/ranger-2.1.0-admin/db/postgres/create_dbversion_catalog.sql
2026-09-23 15:03:29.549 UTC [25478] WARNING: there is no transaction in progress
2026-09-23 15:03:29,556 [I] create_dbversion_catalog.sql file imported successfully
2026-09-23 15:03:29,557 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKC4soy/build/release/bin/postgresql.jar:/tmp/dist-test-taskKC4soy/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.75.148:46271/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_db_version_h') as temp;"
2026-09-23 15:03:29,813 [I] Table x_db_version_h already exists in database 'ranger'
2026-09-23 15:03:29,813 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-23 15:03:29,813 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKC4soy/build/release/bin/postgresql.jar:/tmp/dist-test-taskKC4soy/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.75.148:46271/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'Y';"
2026-09-23 15:03:30,093 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKC4soy/build/release/bin/postgresql.jar:/tmp/dist-test-taskKC4soy/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.75.148:46271/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'CORE_DB_SCHEMA' and active = 'N';"
2026-09-23 15:03:30,337 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKC4soy/build/release/bin/postgresql.jar:/tmp/dist-test-taskKC4soy/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.75.148:46271/ranger -u miniranger -p '********' -noheader -trim -c \; -query "insert into x_db_version_h (version, inst_at, inst_by, updated_at, updated_by,active) values ('CORE_DB_SCHEMA', current_timestamp, 'Ranger 2.1.0', current_timestamp, 'dist-test-slave-cnb0.c.gcp-upstream.internal','N') ;"
2026-09-23 15:03:30,588 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKC4soy/build/release/bin/postgresql.jar:/tmp/dist-test-taskKC4soy/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.75.148:46271/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-23 15:03:30,856 [I] Table x_portal_user does not exist in database ranger
2026-09-23 15:03:30,857 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKC4soy/build/release/bin/postgresql.jar:/tmp/dist-test-taskKC4soy/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.75.148:46271/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-23 15:03:31,091 [I] Table x_policy_ref_group does not exist in database ranger
2026-09-23 15:03:31,091 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKC4soy/build/release/bin/postgresql.jar:/tmp/dist-test-taskKC4soy/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.75.148:46271/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-23 15:03:31,358 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKC4soy/build/release/bin/postgresql.jar:/tmp/dist-test-taskKC4soy/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.75.148:46271/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-23 15:03:31,605 [I] Importing DB schema to database ranger from file: ranger_core_db_postgres.sql
2026-09-23 15:03:31,605 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKC4soy/build/release/bin/postgresql.jar:/tmp/dist-test-taskKC4soy/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.75.148:46271/ranger -u miniranger -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskKC4soy/thirdparty/src/ranger-2.1.0-admin/db/postgres/optimized/current/ranger_core_db_postgres.sql
2026-09-23 15:03:32.666 UTC [25586] WARNING: there is no transaction in progress
2026-09-23 15:03:32.683 UTC [25586] WARNING: there is no transaction in progress
2026-09-23 15:03:32.701 UTC [25586] WARNING: there is no transaction in progress
2026-09-23 15:03:32.719 UTC [25586] WARNING: there is no transaction in progress
2026-09-23 15:03:32.737 UTC [25586] WARNING: there is no transaction in progress
2026-09-23 15:03:32.771 UTC [25586] WARNING: there is no transaction in progress
2026-09-23 15:03:32.828 UTC [25586] WARNING: there is no transaction in progress
2026-09-23 15:03:32.835 UTC [25586] WARNING: there is no transaction in progress
2026-09-23 15:03:32.850 UTC [25586] WARNING: there is no transaction in progress
2026-09-23 15:03:32.861 UTC [25586] WARNING: there is no transaction in progress
2026-09-23 15:03:32.872 UTC [25586] WARNING: there is no transaction in progress
2026-09-23 15:03:32.887 UTC [25586] WARNING: there is no transaction in progress
2026-09-23 15:03:32.897 UTC [25586] WARNING: there is no transaction in progress
2026-09-23 15:03:32.904 UTC [25586] WARNING: there is no transaction in progress
2026-09-23 15:03:32.910 UTC [25586] WARNING: there is no transaction in progress
2026-09-23 15:03:32.917 UTC [25586] WARNING: there is no transaction in progress
2026-09-23 15:03:33,625 [I] ranger_core_db_postgres.sql file imported successfully
2026-09-23 15:03:33,625 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKC4soy/build/release/bin/postgresql.jar:/tmp/dist-test-taskKC4soy/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.75.148:46271/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set inst_by='Ranger 2.1.0' where active='Y' and updated_by='localhost';"
2026-09-23 15:03:33,881 [I] Patches status entries updated from base ranger version to current installed ranger version:Ranger 2.1.0
2026-09-23 15:03:33,882 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKC4soy/build/release/bin/postgresql.jar:/tmp/dist-test-taskKC4soy/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.75.148:46271/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_portal_user') as temp;"
2026-09-23 15:03:34,166 [I] Table x_portal_user already exists in database 'ranger'
2026-09-23 15:03:34,167 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKC4soy/build/release/bin/postgresql.jar:/tmp/dist-test-taskKC4soy/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.75.148:46271/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='ranger' and table_name = 'x_policy_ref_group') as temp;"
2026-09-23 15:03:34,438 [I] Table x_policy_ref_group already exists in database 'ranger'
2026-09-23 15:03:34,438 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKC4soy/build/release/bin/postgresql.jar:/tmp/dist-test-taskKC4soy/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.75.148:46271/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and active = 'Y';"
2026-09-23 15:03:34,681 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKC4soy/build/release/bin/postgresql.jar:/tmp/dist-test-taskKC4soy/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.75.148:46271/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'JAVA_PATCHES' and active = 'Y';"
2026-09-23 15:03:34,931 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKC4soy/build/release/bin/postgresql.jar:/tmp/dist-test-taskKC4soy/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.75.148:46271/ranger -u miniranger -p '********' -noheader -trim -c \; -query "update x_db_version_h set active='Y' where version='CORE_DB_SCHEMA' and active='N' and updated_by='dist-test-slave-cnb0.c.gcp-upstream.internal';"
2026-09-23 15:03:35,219 [I] CORE_DB_SCHEMA import status has been updated
2026-09-23 15:03:35,219 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKC4soy/build/release/bin/postgresql.jar:/tmp/dist-test-taskKC4soy/thirdparty/src/ranger-2.1.0-admin/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.75.148:46271/ranger -u miniranger -p '********' -noheader -trim -c \; -query "select version from x_db_version_h where version = 'DB_PATCHES' and inst_by = 'Ranger 2.1.0' and active = 'Y';"
2026-09-23 15:03:35,480 [I] DB_PATCHES have already been applied
I20260923 15:03:35.486874 13614 mini_ranger.cc:191] Using Ranger class path: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/minicluster-data/ranger-admin:/tmp/dist-test-taskKC4soy/thirdparty/src/ranger-2.1.0-admin/ews/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*:/tmp/dist-test-taskKC4soy/thirdparty/src/hadoop-3.2.0/*:/tmp/dist-test-taskKC4soy/build/release/bin/postgresql.jar:/tmp/dist-test-taskKC4soy/thirdparty/src/ranger-2.1.0-admin/ews/webapp
I20260923 15:03:35.486999 13614 mini_ranger.cc:193] Using host: 127.13.75.148
I20260923 15:03:35.490137 13614 mini_ranger.cc:227] Ranger admin URL: http://127.13.75.148:43305
Sep 23, 2026 3:03:35 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 23, 2026 3:03:35 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 23, 2026 3:03:35 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Deriving webapp folder from catalina.base property. folder=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/minicluster-data/ranger-admin/ews/webapp
Sep 23, 2026 3:03:35 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/minicluster-data/ranger-admin/ews/webapp, webAppName = /
Sep 23, 2026 3:03:35 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/] = path [/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/minicluster-data/ranger-admin/ews/webapp] .....
Sep 23, 2026 3:03:36 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/] = path [/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/minicluster-data/ranger-admin/ews/webapp].
Sep 23, 2026 3:03:36 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : miniranger
I20260923 15:03:36.639472 13614 mini_ranger.cc:160] Time spent starting Ranger: real 8.985s user 0.000s sys 0.009s
Sep 23, 2026 3:03:58 PM com.sun.jersey.api.core.PackagesResourceConfig init
INFO: Scanning for root resource and provider classes in the packages:
org.apache.ranger.rest
org.apache.ranger.common
xa.rest
Sep 23, 2026 3:03:58 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Root resource classes found:
class org.apache.ranger.rest.XAuditREST
class org.apache.ranger.rest.TagREST
class org.apache.ranger.rest.UserREST
class org.apache.ranger.rest.RoleREST
class org.apache.ranger.rest.MetricsREST
class org.apache.ranger.rest.SecurityZoneREST
class org.apache.ranger.rest.ServiceREST
class org.apache.ranger.rest.AssetREST
class org.apache.ranger.rest.PublicAPIsv2
class org.apache.ranger.rest.PublicAPIs
class org.apache.ranger.rest.XUserREST
class org.apache.ranger.rest.XKeyREST
Sep 23, 2026 3:03:58 PM com.sun.jersey.api.core.ScanningResourceConfig logClasses
INFO: Provider classes found:
class org.apache.ranger.common.RangerJAXBContextResolver
Sep 23, 2026 3:03:58 PM com.sun.jersey.spi.spring.container.servlet.SpringServlet getContext
INFO: Using default applicationContext
Sep 23, 2026 3:03:58 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, assetREST, of type org.apache.ranger.rest.AssetREST as a root resource class
Sep 23, 2026 3:03:58 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, metricsREST, of type org.apache.ranger.rest.MetricsREST as a root resource class
Sep 23, 2026 3:03:58 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIs, of type org.apache.ranger.rest.PublicAPIs as a root resource class
Sep 23, 2026 3:03:58 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, publicAPIsv2, of type org.apache.ranger.rest.PublicAPIsv2 as a root resource class
Sep 23, 2026 3:03:58 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, roleREST, of type org.apache.ranger.rest.RoleREST as a root resource class
Sep 23, 2026 3:03:58 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, securityZoneREST, of type org.apache.ranger.rest.SecurityZoneREST as a root resource class
Sep 23, 2026 3:03:58 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, serviceREST, of type org.apache.ranger.rest.ServiceREST as a root resource class
Sep 23, 2026 3:03:58 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, tagREST, of type org.apache.ranger.rest.TagREST as a root resource class
Sep 23, 2026 3:03:58 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, userREST, of type org.apache.ranger.rest.UserREST as a root resource class
Sep 23, 2026 3:03:58 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XAuditREST, of type org.apache.ranger.rest.XAuditREST as a root resource class
Sep 23, 2026 3:03:58 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XKeyREST, of type org.apache.ranger.rest.XKeyREST as a root resource class
Sep 23, 2026 3:03:58 PM com.sun.jersey.spi.spring.container.SpringComponentProviderFactory registerSpringBeans
INFO: Registering Spring bean, XUserREST, of type org.apache.ranger.rest.XUserREST as a root resource class
Sep 23, 2026 3:03:58 PM com.sun.jersey.server.impl.application.WebApplicationImpl _initiate
INFO: Initiating Jersey application, version 'Jersey: 1.19.3 10/24/2016 03:58 PM'
Sep 23, 2026 3:03:59 PM com.sun.jersey.spi.inject.Errors processErrorMessages
WARNING: The following warnings have been detected with resource and/or provider classes:
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInExcel(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInJson(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse,java.lang.Boolean), MUST return a non-void type.
WARNING: A HTTP GET method, public void org.apache.ranger.rest.ServiceREST.getPoliciesInCsv(javax.servlet.http.HttpServletRequest,javax.servlet.http.HttpServletResponse) throws java.io.IOException, MUST return a non-void type.
I20260923 15:04:00.051892 13614 mini_ranger.cc:261] Created Kudu service
WARNING: no policy specified for rangerkms/127.13.75.148@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "rangerkms/127.13.75.148@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal rangerkms/127.13.75.148@KRBTEST.COM with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/krb5kdc/rangerkms_127.13.75.148@KRBTEST.COM.keytab.
Entry for principal rangerkms/127.13.75.148@KRBTEST.COM with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/krb5kdc/rangerkms_127.13.75.148@KRBTEST.COM.keytab.
WARNING: no policy specified for HTTP/127.13.75.148@KRBTEST.COM; defaulting to no policy
add_principal: Principal or policy already exists while creating "HTTP/127.13.75.148@KRBTEST.COM".
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal HTTP/127.13.75.148@KRBTEST.COM with kvno 3, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/krb5kdc/HTTP_127.13.75.148@KRBTEST.COM.keytab.
Entry for principal HTTP/127.13.75.148@KRBTEST.COM with kvno 3, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/krb5kdc/HTTP_127.13.75.148@KRBTEST.COM.keytab.
WARNING: no policy specified for keyadmin@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "keyadmin@KRBTEST.COM" created.
Sep 23 15:04:00 dist-test-slave-cnb0 krb5kdc[25373](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175840, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for keyadmin@KRBTEST.COM:
I20260923 15:04:00.140725 13614 mini_ranger_kms.cc:208] Starting Ranger KMS...
I20260923 15:04:00.150691 13614 mini_ranger_kms.cc:78] Created minirangerkms Postgres user
I20260923 15:04:01.170339 13614 mini_ranger_kms.cc:81] Created rangerkms Postgres database
I20260923 15:04:01.170408 13614 mini_ranger_kms.cc:226] Starting Ranger KMS out of /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/minicluster-data/ranger-kms
I20260923 15:04:01.170421 13614 mini_ranger_kms.cc:227] Using postgres at 127.13.75.148:46271
2026-09-23 15:04:01,332 [I] DB FLAVOR :POSTGRES
2026-09-23 15:04:01,333 [I] --------- Verifying Ranger DB connection ---------
2026-09-23 15:04:01,333 [I] Checking connection
2026-09-23 15:04:01,333 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKC4soy/build/release/bin/postgresql.jar:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.75.148:46271/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "SELECT 1;"
2026-09-23 15:04:01,601 [I] connection success
2026-09-23 15:04:01,601 [I] --------- Verifying Ranger DB tables ---------
2026-09-23 15:04:01,601 [I] Verifying table ranger_masterkey in database rangerkms
2026-09-23 15:04:01,602 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKC4soy/build/release/bin/postgresql.jar:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.75.148:46271/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -query "select * from (select table_name from information_schema.tables where table_catalog='rangerkms' and table_name = 'ranger_masterkey') as temp;"
2026-09-23 15:04:01,848 [I] Table ranger_masterkey does not exist in database rangerkms
2026-09-23 15:04:01,849 [I] --------- Importing Ranger Core DB Schema ---------
2026-09-23 15:04:01,849 [I] Importing db schema to database rangerkms from file: kms_core_db_postgres.sql
2026-09-23 15:04:01,849 [JISQL] /usr/lib/jvm/java-1.8.0-openjdk-amd64/bin/java -cp /tmp/dist-test-taskKC4soy/build/release/bin/postgresql.jar:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/minicluster-data/ranger-kms/jisql/lib/* org.apache.util.sql.Jisql -driver postgresql -cstring jdbc:postgresql://127.13.75.148:46271/rangerkms -u rangerkms -p '********' -noheader -trim -c \; -input /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/minicluster-data/ranger-kms/db/postgres/kms_core_db_postgres.sql
2026-09-23 15:04:02,126 [I] kms_core_db_postgres.sql DB schema imported successfully
I20260923 15:04:02.654243 13614 mini_ranger_kms.cc:310] Created kms service
I20260923 15:04:02.874996 13614 mini_ranger_kms.cc:326] Created kudu user
I20260923 15:04:02.923784 13614 mini_ranger_kms.cc:343] Created rangerkms user
Sep 23 15:04:02 dist-test-slave-cnb0 krb5kdc[25373](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175840, etypes {rep=17 tkt=17 ses=17}, keyadmin@KRBTEST.COM for HTTP/127.13.75.148@KRBTEST.COM
I20260923 15:04:03.358697 13614 mini_ranger_kms.cc:384] Added ranger policy
I20260923 15:04:03.358933 13614 mini_ranger_kms.cc:238] Using RangerKMS classpath: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/minicluster-data/ranger-kms:/tmp/dist-test-taskKC4soy/build/release/bin/postgresql.jar:/tmp/dist-test-taskKC4soy/thirdparty/src/ranger-2.1.0-kms/ews/webapp/WEB-INF/classes/lib/*:/tmp/dist-test-taskKC4soy/thirdparty/src/ranger-2.1.0-kms/ews/webapp/lib/*:/usr/lib/jvm/java-1.8.0-openjdk-amd64/lib/*
I20260923 15:04:03.358958 13614 mini_ranger_kms.cc:240] Using host: 127.13.75.148
I20260923 15:04:03.362706 13614 mini_ranger_kms.cc:276] Ranger KMS PID: 25768
I20260923 15:04:03.362794 13614 mini_ranger_kms.cc:277] Ranger KMS URL: http://127.13.75.148:49985
Sep 23, 2026 3:04:04 PM org.apache.ranger.server.tomcat.EmbeddedServer getKeyManagers
WARNING: Config 'ranger.keystore.file' or 'ranger.service.https.attrib.keystore.file' is not found or contains blank value
Sep 23, 2026 3:04:04 PM org.apache.ranger.server.tomcat.EmbeddedServer getTrustManagers
WARNING: Config 'ranger.truststore.file' is not found or contains blank value!
Sep 23, 2026 3:04:04 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Webapp file =/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/minicluster-data/ranger-kms/ews/webapp, webAppName = /kms
Sep 23, 2026 3:04:04 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Adding webapp [/kms] = path [/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/minicluster-data/ranger-kms/ews/webapp] .....
Sep 23, 2026 3:04:04 PM org.apache.ranger.server.tomcat.EmbeddedServer start
INFO: Finished init of webapp [/kms] = path [/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/minicluster-data/ranger-kms/ews/webapp].
Sep 23, 2026 3:04:04 PM org.apache.ranger.server.tomcat.EmbeddedServer startServer
INFO: Server Name : minirangerkms
Sep 23 15:04:05 dist-test-slave-cnb0 krb5kdc[25373](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790175845, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.13.75.148@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:04:05.658406 13614 mini_ranger_kms.cc:207] Time spent starting Ranger KMS: real 5.518s user 0.013s sys 0.133s
I20260923 15:04:05.658511 13614 mini_ranger_kms.cc:397] {"name":"kuduclusterkey","cipher":"AES/CTR/NoPadding","length":128,"description":"kuduclusterkey"}
I20260923 15:04:05.658560 13614 mini_ranger_kms.cc:401] 127.13.75.148:49985/kms/v1/keys
Sep 23 15:04:05 dist-test-slave-cnb0 krb5kdc[25373](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175845, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.13.75.148@KRBTEST.COM for HTTP/127.13.75.148@KRBTEST.COM
Sep 23 15:04:06 dist-test-slave-cnb0 krb5kdc[25373](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175845, etypes {rep=17 tkt=17 ses=17}, rangerkms/127.13.75.148@KRBTEST.COM for HTTP/127.13.75.148@KRBTEST.COM
Sep 23 15:04:08 dist-test-slave-cnb0 krb5kdc[25373](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175848, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
WARNING: no policy specified for kudu/127.13.75.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.75.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.75.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.190.keytab.
Entry for principal kudu/127.13.75.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.190.keytab.
I20260923 15:04:08.780002 13614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKC4soy/build/release/bin/kudu
/tmp/dist-test-taskKC4soy/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.75.190:34053
--webserver_interface=127.13.75.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.190.keytab
--principal=kudu/127.13.75.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.75.148:42213
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.75.190:34053
--ranger_config_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.13.75.148:49985/kms
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:04:08.865514 25818 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:04:08.865793 25818 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:04:08.865832 25818 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:04:08.867209 25818 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 15:04:08.867280 25818 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:04:08.867303 25818 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:04:08.867324 25818 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 15:04:08.867355 25818 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 15:04:08.869097 25818 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.75.148:42213
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/minicluster-data/master-0/wal
--ranger_kms_url=127.13.75.148:49985/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.13.75.190:34053
--ranger_config_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.190.keytab
--principal=kudu/127.13.75.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.75.190:34053
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.75.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25818
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:04:08.869421 25818 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:04:08.869688 25818 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:04:08.876130 25823 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:04:08.876145 25824 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:04:08.876297 25826 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:04:08.876477 25818 server_base.cc:1034] running on GCE node
I20260923 15:04:08.876875 25818 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:04:08.877277 25818 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:04:08.878484 25818 hybrid_clock.cc:648] HybridClock initialized: now 1790175848878458 us; error 42 us; skew 500 ppm
Sep 23 15:04:08 dist-test-slave-cnb0 krb5kdc[25373](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175848, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:04:08.881258 25818 init.cc:377] Logged in from keytab as kudu/127.13.75.190@KRBTEST.COM (short username kudu)
I20260923 15:04:08.881727 25818 webserver.cc:460] Webserver started at http://127.13.75.190:44315/ using document root <none> and password file <none>
I20260923 15:04:08.881985 25818 fs_manager.cc:359] Metadata directory not provided
I20260923 15:04:08.882047 25818 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:04:08.882210 25818 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 23 15:04:08 dist-test-slave-cnb0 krb5kdc[25373](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175848, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.190@KRBTEST.COM for HTTP/127.13.75.148@KRBTEST.COM
I20260923 15:04:08.945222 25818 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/minicluster-data/master-0/data/instance:
uuid: "dbc551ca623f4882ab0ac309072be9f0"
format_stamp: "Formatted at 2026-09-23 15:04:08 on dist-test-slave-cnb0"
server_key: "ff53f1938947487cb4abbebbc5771e7d"
server_key_iv: "35cedb060c3d7e1d62ee652d3c43940a"
server_key_version: "kuduclusterkey@0"
I20260923 15:04:08.945684 25818 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/minicluster-data/master-0/wal/instance:
uuid: "dbc551ca623f4882ab0ac309072be9f0"
format_stamp: "Formatted at 2026-09-23 15:04:08 on dist-test-slave-cnb0"
server_key: "ff53f1938947487cb4abbebbc5771e7d"
server_key_iv: "35cedb060c3d7e1d62ee652d3c43940a"
server_key_version: "kuduclusterkey@0"
I20260923 15:04:08.947630 25818 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260923 15:04:08.962915 25833 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:04:08.963192 25818 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:04:08.963389 25818 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/minicluster-data/master-0/data,/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/minicluster-data/master-0/wal
uuid: "dbc551ca623f4882ab0ac309072be9f0"
format_stamp: "Formatted at 2026-09-23 15:04:08 on dist-test-slave-cnb0"
server_key: "ff53f1938947487cb4abbebbc5771e7d"
server_key_iv: "35cedb060c3d7e1d62ee652d3c43940a"
server_key_version: "kuduclusterkey@0"
I20260923 15:04:08.963500 25818 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:04:08.980067 25818 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:04:08.982590 25818 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:04:08.982752 25818 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:04:08.987991 25818 rpc_server.cc:307] RPC server started. Bound to: 127.13.75.190:34053
I20260923 15:04:08.988108 25885 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.75.190:34053 every 8 connection(s)
I20260923 15:04:08.988452 25818 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/minicluster-data/master-0/data/info.pb
I20260923 15:04:08.989256 25886 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:04:08.991984 25886 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dbc551ca623f4882ab0ac309072be9f0: Bootstrap starting.
I20260923 15:04:08.992694 25886 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P dbc551ca623f4882ab0ac309072be9f0: Neither blocks nor log segments found. Creating new log.
I20260923 15:04:08.993038 25886 log.cc:824] T 00000000000000000000000000000000 P dbc551ca623f4882ab0ac309072be9f0: Log is configured to *not* fsync() on all Append() calls
I20260923 15:04:08.993870 25886 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dbc551ca623f4882ab0ac309072be9f0: No bootstrap required, opened a new log
I20260923 15:04:08.997013 13614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKC4soy/build/release/bin/kudu as pid 25818
I20260923 15:04:08.996822 25886 raft_consensus.cc:348] T 00000000000000000000000000000000 P dbc551ca623f4882ab0ac309072be9f0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbc551ca623f4882ab0ac309072be9f0" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 34053 } }
I20260923 15:04:08.997084 25886 raft_consensus.cc:374] T 00000000000000000000000000000000 P dbc551ca623f4882ab0ac309072be9f0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:04:08.997126 13614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/minicluster-data/master-0/wal/instance
I20260923 15:04:08.997140 25886 raft_consensus.cc:729] T 00000000000000000000000000000000 P dbc551ca623f4882ab0ac309072be9f0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbc551ca623f4882ab0ac309072be9f0, State: Initialized, Role: FOLLOWER
I20260923 15:04:08.997318 13614 external_mini_cluster.cc:1428] Setting key d579dbb9a36d62569e819491ef5d3457
I20260923 15:04:08.997438 25886 consensus_queue.cc:260] T 00000000000000000000000000000000 P dbc551ca623f4882ab0ac309072be9f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbc551ca623f4882ab0ac309072be9f0" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 34053 } }
I20260923 15:04:08.997540 25886 raft_consensus.cc:388] T 00000000000000000000000000000000 P dbc551ca623f4882ab0ac309072be9f0 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 15:04:08.997614 25886 raft_consensus.cc:482] T 00000000000000000000000000000000 P dbc551ca623f4882ab0ac309072be9f0 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 15:04:08.997686 25886 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dbc551ca623f4882ab0ac309072be9f0 [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:04:08.998847 25886 raft_consensus.cc:504] T 00000000000000000000000000000000 P dbc551ca623f4882ab0ac309072be9f0 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbc551ca623f4882ab0ac309072be9f0" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 34053 } }
I20260923 15:04:08.999137 25886 leader_election.cc:302] T 00000000000000000000000000000000 P dbc551ca623f4882ab0ac309072be9f0 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dbc551ca623f4882ab0ac309072be9f0; no voters:
I20260923 15:04:08.999444 25886 leader_election.cc:288] T 00000000000000000000000000000000 P dbc551ca623f4882ab0ac309072be9f0 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 15:04:08.999485 25891 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dbc551ca623f4882ab0ac309072be9f0 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 15:04:08.999730 25891 raft_consensus.cc:686] T 00000000000000000000000000000000 P dbc551ca623f4882ab0ac309072be9f0 [term 1 LEADER]: Becoming Leader. State: Replica: dbc551ca623f4882ab0ac309072be9f0, State: Running, Role: LEADER
I20260923 15:04:08.999862 25891 consensus_queue.cc:237] T 00000000000000000000000000000000 P dbc551ca623f4882ab0ac309072be9f0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbc551ca623f4882ab0ac309072be9f0" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 34053 } }
I20260923 15:04:09.000226 25891 sys_catalog.cc:455] T 00000000000000000000000000000000 P dbc551ca623f4882ab0ac309072be9f0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "dbc551ca623f4882ab0ac309072be9f0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbc551ca623f4882ab0ac309072be9f0" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 34053 } } }
I20260923 15:04:09.000389 25891 sys_catalog.cc:458] T 00000000000000000000000000000000 P dbc551ca623f4882ab0ac309072be9f0 [sys.catalog]: This master's current role is: LEADER
I20260923 15:04:09.000560 25891 sys_catalog.cc:455] T 00000000000000000000000000000000 P dbc551ca623f4882ab0ac309072be9f0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dbc551ca623f4882ab0ac309072be9f0. Latest consensus state: current_term: 1 leader_uuid: "dbc551ca623f4882ab0ac309072be9f0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbc551ca623f4882ab0ac309072be9f0" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 34053 } } }
I20260923 15:04:09.000662 25891 sys_catalog.cc:458] T 00000000000000000000000000000000 P dbc551ca623f4882ab0ac309072be9f0 [sys.catalog]: This master's current role is: LEADER
I20260923 15:04:08.999869 25886 sys_catalog.cc:564] T 00000000000000000000000000000000 P dbc551ca623f4882ab0ac309072be9f0 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 15:04:09.002238 25886 ranger_client.cc:318] Using new properties file: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260923 15:04:09.005008 25895 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 15:04:09.006569 25895 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Sep 23 15:04:09 dist-test-slave-cnb0 krb5kdc[25373](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175848, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.75.190@KRBTEST.COM
I20260923 15:04:09.014631 25890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:08.997859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32842 (local address 127.13.75.190:34053)
0923 15:04:08.998128 (+ 269us) server_negotiation.cc:206] Beginning negotiation
0923 15:04:08.998141 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0923 15:04:08.998185 (+ 44us) server_negotiation.cc:407] Connection header received
0923 15:04:08.998331 (+ 146us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:04:08.998338 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:04:08.998418 (+ 80us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:04:08.998588 (+ 170us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:04:08.999747 (+ 1159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:04:09.001518 (+ 1771us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:04:09.002466 (+ 948us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:04:09.006135 (+ 3669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:04:09.006955 (+ 820us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:04:09.010141 (+ 3186us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:04:09.010147 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:04:09.010147 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:04:09.010162 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:04:09.012391 (+ 2229us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:04:09.012894 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:04:09.012897 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:04:09.012897 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:04:09.013018 (+ 121us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:04:09.013241 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:04:09.013241 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:04:09.013242 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:04:09.013586 (+ 344us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:04:09.014144 (+ 558us) server_negotiation.cc:1035] Waiting for connection context
0923 15:04:09.014218 (+ 74us) server_negotiation.cc:299] Negotiation successful
0923 15:04:09.014365 (+ 147us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":206,"thread_start_us":144,"threads_started":1}
I20260923 15:04:09.016084 25895 catalog_manager.cc:1347] Generated new cluster ID: 2414b62f39144bfcbd3161383bbe221b
I20260923 15:04:09.016144 25895 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 15:04:09.041980 25895 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 15:04:09.042865 25895 catalog_manager.cc:1504] Loading token signing keys...
I20260923 15:04:09.047526 25895 catalog_manager.cc:5868] T 00000000000000000000000000000000 P dbc551ca623f4882ab0ac309072be9f0: Generated new TSK 0
I20260923 15:04:09.047842 25895 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.75.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.75.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.75.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.129.keytab.
Entry for principal kudu/127.13.75.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.129.keytab.
I20260923 15:04:09.783229 13614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKC4soy/build/release/bin/kudu
/tmp/dist-test-taskKC4soy/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.13.75.148:49985/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.75.129:0
--local_ip_for_outbound_sockets=127.13.75.129
--webserver_interface=127.13.75.129
--webserver_port=0
--tserver_master_addrs=127.13.75.190:34053
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.129.keytab
--principal=kudu/127.13.75.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.75.148:42213
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:04:09.904376 25931 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:04:09.904654 25931 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:04:09.904690 25931 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:04:09.906438 25931 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:04:09.906510 25931 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:04:09.906553 25931 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.75.129
I20260923 15:04:09.908802 25931 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.75.148:42213
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/minicluster-data/ts-0/wal
--ranger_kms_url=127.13.75.148:49985/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.129.keytab
--principal=kudu/127.13.75.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.75.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.75.129
--webserver_port=0
--tserver_master_addrs=127.13.75.190:34053
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.25931
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.75.129
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:04:09.909093 25931 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:04:09.909399 25931 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:04:09.915984 25937 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:04:09.915974 25936 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:04:09.916275 25931 server_base.cc:1034] running on GCE node
W20260923 15:04:09.915974 25939 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:04:09.916626 25931 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:04:09.916940 25931 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:04:09.918133 25931 hybrid_clock.cc:648] HybridClock initialized: now 1790175849918121 us; error 31 us; skew 500 ppm
Sep 23 15:04:09 dist-test-slave-cnb0 krb5kdc[25373](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175849, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:04:09.921072 25931 init.cc:377] Logged in from keytab as kudu/127.13.75.129@KRBTEST.COM (short username kudu)
I20260923 15:04:09.921612 25931 webserver.cc:460] Webserver started at http://127.13.75.129:42981/ using document root <none> and password file <none>
I20260923 15:04:09.921907 25931 fs_manager.cc:359] Metadata directory not provided
I20260923 15:04:09.921980 25931 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:04:09.922133 25931 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 23 15:04:09 dist-test-slave-cnb0 krb5kdc[25373](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175849, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.129@KRBTEST.COM for HTTP/127.13.75.148@KRBTEST.COM
I20260923 15:04:09.936218 25931 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/minicluster-data/ts-0/data/instance:
uuid: "2e9e6fe7c03c4f2680c9679a16e60436"
format_stamp: "Formatted at 2026-09-23 15:04:09 on dist-test-slave-cnb0"
server_key: "2d8f9e57a3fee1a3f3e99b98919f62b8"
server_key_iv: "b1ae33ff9ab1782dc9106c36574a8100"
server_key_version: "kuduclusterkey@0"
I20260923 15:04:09.936816 25931 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/minicluster-data/ts-0/wal/instance:
uuid: "2e9e6fe7c03c4f2680c9679a16e60436"
format_stamp: "Formatted at 2026-09-23 15:04:09 on dist-test-slave-cnb0"
server_key: "2d8f9e57a3fee1a3f3e99b98919f62b8"
server_key_iv: "b1ae33ff9ab1782dc9106c36574a8100"
server_key_version: "kuduclusterkey@0"
I20260923 15:04:09.938787 25931 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.001s sys 0.000s
I20260923 15:04:09.954949 25948 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:04:09.955507 25931 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:04:09.955615 25931 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/minicluster-data/ts-0/wal
uuid: "2e9e6fe7c03c4f2680c9679a16e60436"
format_stamp: "Formatted at 2026-09-23 15:04:09 on dist-test-slave-cnb0"
server_key: "2d8f9e57a3fee1a3f3e99b98919f62b8"
server_key_iv: "b1ae33ff9ab1782dc9106c36574a8100"
server_key_version: "kuduclusterkey@0"
I20260923 15:04:09.955701 25931 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:04:09.986618 25931 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:04:09.989430 25931 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:04:09.989691 25931 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:04:09.990046 25931 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 15:04:09.990496 25931 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 15:04:09.990592 25931 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 15:04:09.990660 25931 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 15:04:09.990844 25931 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 15:04:09.998694 25931 rpc_server.cc:307] RPC server started. Bound to: 127.13.75.129:35677
I20260923 15:04:09.999430 25931 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/minicluster-data/ts-0/data/info.pb
I20260923 15:04:10.000655 26061 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.75.129:35677 every 8 connection(s)
I20260923 15:04:10.001063 13614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKC4soy/build/release/bin/kudu as pid 25931
I20260923 15:04:10.001178 13614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/minicluster-data/ts-0/wal/instance
I20260923 15:04:10.001380 13614 external_mini_cluster.cc:1428] Setting key 07a5b47d89d4cb89d9c3b1b2bbb54892
Sep 23 15:04:10 dist-test-slave-cnb0 krb5kdc[25373](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175849, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.129@KRBTEST.COM for kudu/127.13.75.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.13.75.130@KRBTEST.COM; defaulting to no policy
I20260923 15:04:10.014950 26064 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:10.001443 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:34053 (local address 127.13.75.129:36159)
0923 15:04:10.001651 (+ 208us) negotiation.cc:107] Waiting for socket to connect
0923 15:04:10.002052 (+ 401us) client_negotiation.cc:174] Beginning negotiation
0923 15:04:10.002798 (+ 746us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:04:10.003268 (+ 470us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:04:10.003273 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:04:10.003561 (+ 288us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:04:10.003886 (+ 325us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:04:10.003892 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:04:10.004967 (+ 1075us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:04:10.004968 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:04:10.006507 (+ 1539us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:04:10.006511 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:04:10.007168 (+ 657us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:04:10.007169 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:04:10.007336 (+ 167us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:04:10.008066 (+ 730us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:04:10.008080 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:04:10.010017 (+ 1937us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:04:10.012379 (+ 2362us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:04:10.012390 (+ 11us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:04:10.012392 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:04:10.012659 (+ 267us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:04:10.012873 (+ 214us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:04:10.012874 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:04:10.012874 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:04:10.013003 (+ 129us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:04:10.014114 (+ 1111us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:04:10.014121 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:04:10.014336 (+ 215us) client_negotiation.cc:769] Sending connection context
0923 15:04:10.014474 (+ 138us) client_negotiation.cc:240] Negotiation successful
0923 15:04:10.014653 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":86,"threads_started":1}
I20260923 15:04:10.015923 26065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:10.001474 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.129:36159 (local address 127.13.75.190:34053)
0923 15:04:10.003025 (+ 1551us) server_negotiation.cc:206] Beginning negotiation
0923 15:04:10.003028 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:04:10.003035 (+ 7us) server_negotiation.cc:407] Connection header received
0923 15:04:10.003058 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:04:10.003059 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:04:10.003075 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:04:10.003136 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:04:10.003991 (+ 855us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:04:10.004830 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:04:10.006623 (+ 1793us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:04:10.007040 (+ 417us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:04:10.007704 (+ 664us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:04:10.010156 (+ 2452us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:04:10.010157 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:04:10.010158 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:04:10.010167 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:04:10.012248 (+ 2081us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:04:10.012737 (+ 489us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:04:10.012738 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:04:10.012738 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:04:10.012787 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:04:10.013080 (+ 293us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:04:10.013081 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:04:10.013081 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:04:10.013213 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:04:10.013293 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0923 15:04:10.015688 (+ 2395us) server_negotiation.cc:299] Negotiation successful
0923 15:04:10.015801 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1518,"thread_start_us":87,"threads_started":1}
I20260923 15:04:10.016548 26062 heartbeater.cc:347] Connected to a master server at 127.13.75.190:34053
I20260923 15:04:10.016654 26062 heartbeater.cc:464] Registering TS with master...
I20260923 15:04:10.016885 26062 heartbeater.cc:511] Master 127.13.75.190:34053 requested a full tablet report, sending...
I20260923 15:04:10.017568 25847 ts_manager.cc:194] Registered new tserver with Master: 2e9e6fe7c03c4f2680c9679a16e60436 (127.13.75.129:35677)
I20260923 15:04:10.018561 25847 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.75.129@KRBTEST.COM'} at 127.13.75.129:36159
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.75.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.75.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.130.keytab.
Entry for principal kudu/127.13.75.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.130.keytab.
I20260923 15:04:10.044118 13614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKC4soy/build/release/bin/kudu
/tmp/dist-test-taskKC4soy/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.13.75.148:49985/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.75.130:0
--local_ip_for_outbound_sockets=127.13.75.130
--webserver_interface=127.13.75.130
--webserver_port=0
--tserver_master_addrs=127.13.75.190:34053
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.130.keytab
--principal=kudu/127.13.75.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.75.148:42213
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 23 15:04:10 dist-test-slave-cnb0 krb5kdc[25373](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790175850, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
W20260923 15:04:10.175483 26068 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:04:10.175777 26068 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:04:10.175814 26068 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:04:10.177891 26068 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:04:10.178066 26068 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:04:10.178169 26068 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.75.130
I20260923 15:04:10.180953 26068 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.75.148:42213
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/minicluster-data/ts-1/wal
--ranger_kms_url=127.13.75.148:49985/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.130.keytab
--principal=kudu/127.13.75.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.75.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.75.130
--webserver_port=0
--tserver_master_addrs=127.13.75.190:34053
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26068
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.75.130
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:04:10.181273 26068 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:04:10.181628 26068 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:04:10.188892 26076 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:04:10.188967 26075 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:04:10.189396 26078 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:04:10.190116 26068 server_base.cc:1034] running on GCE node
I20260923 15:04:10.190347 26068 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:04:10.190620 26068 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:04:10.191768 26068 hybrid_clock.cc:648] HybridClock initialized: now 1790175850191748 us; error 36 us; skew 500 ppm
Sep 23 15:04:10 dist-test-slave-cnb0 krb5kdc[25373](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175850, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:04:10.195088 26068 init.cc:377] Logged in from keytab as kudu/127.13.75.130@KRBTEST.COM (short username kudu)
I20260923 15:04:10.195696 26068 webserver.cc:460] Webserver started at http://127.13.75.130:42951/ using document root <none> and password file <none>
I20260923 15:04:10.195999 26068 fs_manager.cc:359] Metadata directory not provided
I20260923 15:04:10.196066 26068 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:04:10.196195 26068 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 23 15:04:10 dist-test-slave-cnb0 krb5kdc[25373](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175850, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.130@KRBTEST.COM for HTTP/127.13.75.148@KRBTEST.COM
I20260923 15:04:10.217738 26068 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/minicluster-data/ts-1/data/instance:
uuid: "5d5b6b7baf20426488ded3114f167e34"
format_stamp: "Formatted at 2026-09-23 15:04:10 on dist-test-slave-cnb0"
server_key: "cd0e229d7ecf7560e84311e6abe99573"
server_key_iv: "3fc94cde5e401aef3482d9439917762a"
server_key_version: "kuduclusterkey@0"
I20260923 15:04:10.218382 26068 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/minicluster-data/ts-1/wal/instance:
uuid: "5d5b6b7baf20426488ded3114f167e34"
format_stamp: "Formatted at 2026-09-23 15:04:10 on dist-test-slave-cnb0"
server_key: "cd0e229d7ecf7560e84311e6abe99573"
server_key_iv: "3fc94cde5e401aef3482d9439917762a"
server_key_version: "kuduclusterkey@0"
I20260923 15:04:10.227391 26068 fs_manager.cc:691] Time spent creating directory manager: real 0.009s user 0.000s sys 0.002s
I20260923 15:04:10.252974 26086 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:04:10.254050 26068 fs_manager.cc:725] Time spent opening block manager: real 0.006s user 0.001s sys 0.000s
I20260923 15:04:10.254173 26068 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/minicluster-data/ts-1/wal
uuid: "5d5b6b7baf20426488ded3114f167e34"
format_stamp: "Formatted at 2026-09-23 15:04:10 on dist-test-slave-cnb0"
server_key: "cd0e229d7ecf7560e84311e6abe99573"
server_key_iv: "3fc94cde5e401aef3482d9439917762a"
server_key_version: "kuduclusterkey@0"
I20260923 15:04:10.254289 26068 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:04:10.271137 26068 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:04:10.273870 26068 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:04:10.274113 26068 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:04:10.274446 26068 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 15:04:10.274955 26068 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 15:04:10.275064 26068 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 15:04:10.275135 26068 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 15:04:10.275190 26068 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 15:04:10.282320 26068 rpc_server.cc:307] RPC server started. Bound to: 127.13.75.130:44109
I20260923 15:04:10.283183 26068 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/minicluster-data/ts-1/data/info.pb
I20260923 15:04:10.284368 13614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKC4soy/build/release/bin/kudu as pid 26068
I20260923 15:04:10.284487 13614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/minicluster-data/ts-1/wal/instance
I20260923 15:04:10.284669 13614 external_mini_cluster.cc:1428] Setting key e72408b754e55f4ac2693bcc81c3bf59
I20260923 15:04:10.288336 26199 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.75.130:44109 every 8 connection(s)
WARNING: no policy specified for kudu/127.13.75.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.75.131@KRBTEST.COM" created.
Sep 23 15:04:10 dist-test-slave-cnb0 krb5kdc[25373](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175850, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.130@KRBTEST.COM for kudu/127.13.75.190@KRBTEST.COM
I20260923 15:04:10.317483 26203 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:10.289021 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:34053 (local address 127.13.75.130:52243)
0923 15:04:10.292986 (+ 3965us) negotiation.cc:107] Waiting for socket to connect
0923 15:04:10.293039 (+ 53us) client_negotiation.cc:174] Beginning negotiation
0923 15:04:10.293805 (+ 766us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:04:10.294246 (+ 441us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:04:10.294251 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:04:10.294556 (+ 305us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:04:10.294866 (+ 310us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:04:10.294873 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:04:10.302695 (+ 7822us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:04:10.302697 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:04:10.304316 (+ 1619us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:04:10.304323 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:04:10.305056 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:04:10.305058 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:04:10.305237 (+ 179us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:04:10.305946 (+ 709us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:04:10.305960 (+ 14us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:04:10.311983 (+ 6023us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:04:10.315325 (+ 3342us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:04:10.315340 (+ 15us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:04:10.315343 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:04:10.315749 (+ 406us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:04:10.316016 (+ 267us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:04:10.316016 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:04:10.316017 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:04:10.316181 (+ 164us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:04:10.316688 (+ 507us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:04:10.316696 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:04:10.316880 (+ 184us) client_negotiation.cc:769] Sending connection context
0923 15:04:10.316988 (+ 108us) client_negotiation.cc:240] Negotiation successful
0923 15:04:10.317165 (+ 177us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3907,"thread_start_us":110,"threads_started":1}
I20260923 15:04:10.317955 26065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:10.289365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.130:52243 (local address 127.13.75.190:34053)
0923 15:04:10.289418 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0923 15:04:10.289421 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:04:10.293164 (+ 3743us) server_negotiation.cc:407] Connection header received
0923 15:04:10.293965 (+ 801us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:04:10.293967 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:04:10.293983 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:04:10.294020 (+ 37us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:04:10.300368 (+ 6348us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:04:10.301355 (+ 987us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:04:10.304486 (+ 3131us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:04:10.304849 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:04:10.304909 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:04:10.312163 (+ 7254us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:04:10.312165 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:04:10.312166 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:04:10.312177 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:04:10.315174 (+ 2997us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:04:10.315862 (+ 688us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:04:10.315864 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:04:10.315864 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:04:10.315934 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:04:10.316299 (+ 365us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:04:10.316301 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:04:10.316301 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:04:10.316572 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:04:10.316656 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0923 15:04:10.317757 (+ 1101us) server_negotiation.cc:299] Negotiation successful
0923 15:04:10.317860 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260923 15:04:10.318499 26200 heartbeater.cc:347] Connected to a master server at 127.13.75.190:34053
I20260923 15:04:10.318601 26200 heartbeater.cc:464] Registering TS with master...
I20260923 15:04:10.318804 26200 heartbeater.cc:511] Master 127.13.75.190:34053 requested a full tablet report, sending...
I20260923 15:04:10.319216 25847 ts_manager.cc:194] Registered new tserver with Master: 5d5b6b7baf20426488ded3114f167e34 (127.13.75.130:44109)
I20260923 15:04:10.319718 25847 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.75.130@KRBTEST.COM'} at 127.13.75.130:52243
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.75.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.131.keytab.
Entry for principal kudu/127.13.75.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.131.keytab.
I20260923 15:04:10.338837 13614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKC4soy/build/release/bin/kudu
/tmp/dist-test-taskKC4soy/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.13.75.148:49985/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.75.131:0
--local_ip_for_outbound_sockets=127.13.75.131
--webserver_interface=127.13.75.131
--webserver_port=0
--tserver_master_addrs=127.13.75.190:34053
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.131.keytab
--principal=kudu/127.13.75.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.75.148:42213
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:04:10.461251 26205 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:04:10.461553 26205 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:04:10.461587 26205 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:04:10.463317 26205 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:04:10.463384 26205 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:04:10.463426 26205 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.75.131
I20260923 15:04:10.465576 26205 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.75.148:42213
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/minicluster-data/ts-2/wal
--ranger_kms_url=127.13.75.148:49985/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.131.keytab
--principal=kudu/127.13.75.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.75.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.75.131
--webserver_port=0
--tserver_master_addrs=127.13.75.190:34053
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26205
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.75.131
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:04:10.465865 26205 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:04:10.466148 26205 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:04:10.472519 26211 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:04:10.472620 26213 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:04:10.472786 26210 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:04:10.473860 26205 server_base.cc:1034] running on GCE node
I20260923 15:04:10.474138 26205 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:04:10.474448 26205 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:04:10.475674 26205 hybrid_clock.cc:648] HybridClock initialized: now 1790175850475656 us; error 27 us; skew 500 ppm
Sep 23 15:04:10 dist-test-slave-cnb0 krb5kdc[25373](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175850, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:04:10.479429 26205 init.cc:377] Logged in from keytab as kudu/127.13.75.131@KRBTEST.COM (short username kudu)
I20260923 15:04:10.480580 26205 webserver.cc:460] Webserver started at http://127.13.75.131:46697/ using document root <none> and password file <none>
I20260923 15:04:10.480970 26205 fs_manager.cc:359] Metadata directory not provided
I20260923 15:04:10.481060 26205 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:04:10.481240 26205 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
Sep 23 15:04:10 dist-test-slave-cnb0 krb5kdc[25373](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175850, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.131@KRBTEST.COM for HTTP/127.13.75.148@KRBTEST.COM
I20260923 15:04:10.499702 26205 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/minicluster-data/ts-2/data/instance:
uuid: "2301dc7034fb46b8bb6165fd0e620ba2"
format_stamp: "Formatted at 2026-09-23 15:04:10 on dist-test-slave-cnb0"
server_key: "7b9576c7abc15debf9e99b7994dee75b"
server_key_iv: "2d0a878779252fda70a6ad68f362d6fd"
server_key_version: "kuduclusterkey@0"
I20260923 15:04:10.500216 26205 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/minicluster-data/ts-2/wal/instance:
uuid: "2301dc7034fb46b8bb6165fd0e620ba2"
format_stamp: "Formatted at 2026-09-23 15:04:10 on dist-test-slave-cnb0"
server_key: "7b9576c7abc15debf9e99b7994dee75b"
server_key_iv: "2d0a878779252fda70a6ad68f362d6fd"
server_key_version: "kuduclusterkey@0"
I20260923 15:04:10.507161 26205 fs_manager.cc:691] Time spent creating directory manager: real 0.007s user 0.001s sys 0.000s
I20260923 15:04:10.531076 26220 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:04:10.531298 26205 fs_manager.cc:725] Time spent opening block manager: real 0.008s user 0.001s sys 0.000s
I20260923 15:04:10.531406 26205 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/minicluster-data/ts-2/wal
uuid: "2301dc7034fb46b8bb6165fd0e620ba2"
format_stamp: "Formatted at 2026-09-23 15:04:10 on dist-test-slave-cnb0"
server_key: "7b9576c7abc15debf9e99b7994dee75b"
server_key_iv: "2d0a878779252fda70a6ad68f362d6fd"
server_key_version: "kuduclusterkey@0"
I20260923 15:04:10.531522 26205 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:04:10.573999 26205 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:04:10.576879 26205 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:04:10.577064 26205 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:04:10.577346 26205 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 15:04:10.577872 26205 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 15:04:10.577968 26205 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 15:04:10.578012 26205 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 15:04:10.578037 26205 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 15:04:10.586249 26205 rpc_server.cc:307] RPC server started. Bound to: 127.13.75.131:39955
I20260923 15:04:10.586720 26205 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/minicluster-data/ts-2/data/info.pb
I20260923 15:04:10.590955 26333 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.75.131:39955 every 8 connection(s)
I20260923 15:04:10.593330 13614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKC4soy/build/release/bin/kudu as pid 26205
I20260923 15:04:10.593449 13614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/minicluster-data/ts-2/wal/instance
I20260923 15:04:10.593650 13614 external_mini_cluster.cc:1428] Setting key 51bf5ced81eb77c1d3c3b153bef4cd71
Sep 23 15:04:10 dist-test-slave-cnb0 krb5kdc[25373](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175850, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.131@KRBTEST.COM for kudu/127.13.75.190@KRBTEST.COM
I20260923 15:04:10.605713 26065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:10.591754 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.131:54907 (local address 127.13.75.190:34053)
0923 15:04:10.591804 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0923 15:04:10.591806 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:04:10.592118 (+ 312us) server_negotiation.cc:407] Connection header received
0923 15:04:10.593332 (+ 1214us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:04:10.593334 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:04:10.593350 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:04:10.593520 (+ 170us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:04:10.594949 (+ 1429us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:04:10.595829 (+ 880us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:04:10.597520 (+ 1691us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:04:10.598014 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:04:10.598103 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:04:10.601100 (+ 2997us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:04:10.601102 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:04:10.601102 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:04:10.601111 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:04:10.603192 (+ 2081us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:04:10.603832 (+ 640us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:04:10.603833 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:04:10.603834 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:04:10.603905 (+ 71us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:04:10.604196 (+ 291us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:04:10.604197 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:04:10.604197 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:04:10.604342 (+ 145us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:04:10.604403 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0923 15:04:10.605510 (+ 1107us) server_negotiation.cc:299] Negotiation successful
0923 15:04:10.605607 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260923 15:04:10.606280 26336 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:10.591736 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:34053 (local address 127.13.75.131:54907)
0923 15:04:10.591934 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0923 15:04:10.591988 (+ 54us) client_negotiation.cc:174] Beginning negotiation
0923 15:04:10.592976 (+ 988us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:04:10.593978 (+ 1002us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:04:10.593985 (+ 7us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:04:10.594441 (+ 456us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:04:10.594775 (+ 334us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:04:10.594784 (+ 9us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:04:10.595947 (+ 1163us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:04:10.595948 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:04:10.597408 (+ 1460us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:04:10.597412 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:04:10.598145 (+ 733us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:04:10.598146 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:04:10.598314 (+ 168us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:04:10.599102 (+ 788us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:04:10.599113 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:04:10.600885 (+ 1772us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:04:10.603303 (+ 2418us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:04:10.603313 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:04:10.603316 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:04:10.603728 (+ 412us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:04:10.603983 (+ 255us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:04:10.603984 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:04:10.603984 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:04:10.604122 (+ 138us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:04:10.604458 (+ 336us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:04:10.604466 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:04:10.604649 (+ 183us) client_negotiation.cc:769] Sending connection context
0923 15:04:10.605832 (+ 1183us) client_negotiation.cc:240] Negotiation successful
0923 15:04:10.606024 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":111,"threads_started":1}
I20260923 15:04:10.606805 26334 heartbeater.cc:347] Connected to a master server at 127.13.75.190:34053
I20260923 15:04:10.606956 26334 heartbeater.cc:464] Registering TS with master...
I20260923 15:04:10.607187 26334 heartbeater.cc:511] Master 127.13.75.190:34053 requested a full tablet report, sending...
I20260923 15:04:10.607671 25847 ts_manager.cc:194] Registered new tserver with Master: 2301dc7034fb46b8bb6165fd0e620ba2 (127.13.75.131:39955)
I20260923 15:04:10.608530 25847 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.75.131@KRBTEST.COM'} at 127.13.75.131:54907
I20260923 15:04:10.616501 13614 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260923 15:04:10.617249 13614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKC4soy/build/release/bin/kudu with pid 25931
I20260923 15:04:10.625763 13614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKC4soy/build/release/bin/kudu with pid 26068
I20260923 15:04:10.633105 13614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKC4soy/build/release/bin/kudu with pid 26205
I20260923 15:04:10.644764 13614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKC4soy/build/release/bin/kudu with pid 25818
I20260923 15:04:10.651830 13614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKC4soy/build/release/bin/kudu
/tmp/dist-test-taskKC4soy/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.75.190:34053
--webserver_interface=127.13.75.190
--webserver_port=44315
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.190.keytab
--principal=kudu/127.13.75.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.75.148:42213
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.75.190:34053
--ranger_config_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/minicluster-data/ranger-client
--trusted_user_acl=test-admin
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.13.75.148:49985/kms
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.13.75.148:49985/kms with env {KRB5CCNAME=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:04:10.747597 26337 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:04:10.748061 26337 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:04:10.748107 26337 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:04:10.749648 26337 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 15:04:10.749712 26337 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:04:10.749728 26337 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:04:10.749742 26337 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 15:04:10.749753 26337 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 15:04:10.751715 26337 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.75.148:42213
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/minicluster-data/master-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.13.75.148:49985/kms
--trusted_user_acl=<redacted>
--ipki_ca_key_size=768
--master_addresses=127.13.75.190:34053
--ranger_config_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/minicluster-data/ranger-client
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.190.keytab
--principal=kudu/127.13.75.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.75.190:34053
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.75.190
--webserver_port=44315
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26337
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:04:10.751994 26337 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:04:10.752259 26337 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:04:10.757745 26342 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:04:10.757969 26337 server_base.cc:1034] running on GCE node
W20260923 15:04:10.757745 26345 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:04:10.757783 26343 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:04:10.758320 26337 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:04:10.758639 26337 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:04:10.759804 26337 hybrid_clock.cc:648] HybridClock initialized: now 1790175850759772 us; error 65 us; skew 500 ppm
Sep 23 15:04:10 dist-test-slave-cnb0 krb5kdc[25373](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175850, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:04:10.762413 26337 init.cc:377] Logged in from keytab as kudu/127.13.75.190@KRBTEST.COM (short username kudu)
I20260923 15:04:10.762866 26337 webserver.cc:460] Webserver started at http://127.13.75.190:44315/ using document root <none> and password file <none>
I20260923 15:04:10.763146 26337 fs_manager.cc:359] Metadata directory not provided
I20260923 15:04:10.763224 26337 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 23 15:04:10 dist-test-slave-cnb0 krb5kdc[25373](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175850, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.190@KRBTEST.COM for HTTP/127.13.75.148@KRBTEST.COM
I20260923 15:04:10.821731 26337 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.001s
I20260923 15:04:10.822649 26353 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:04:10.823012 26337 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:04:10.823122 26337 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/minicluster-data/master-0/data,/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/minicluster-data/master-0/wal
uuid: "dbc551ca623f4882ab0ac309072be9f0"
format_stamp: "Formatted at 2026-09-23 15:04:08 on dist-test-slave-cnb0"
server_key: "ff53f1938947487cb4abbebbc5771e7d"
server_key_iv: "35cedb060c3d7e1d62ee652d3c43940a"
server_key_version: "kuduclusterkey@0"
I20260923 15:04:10.823604 26337 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:04:10.850557 26337 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:04:10.853085 26337 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:04:10.853214 26337 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:04:10.857539 26337 rpc_server.cc:307] RPC server started. Bound to: 127.13.75.190:34053
I20260923 15:04:10.857580 26405 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.75.190:34053 every 8 connection(s)
I20260923 15:04:10.857980 26337 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/minicluster-data/master-0/data/info.pb
I20260923 15:04:10.859342 26406 sys_catalog.cc:263] Verifying existing consensus state
I20260923 15:04:10.859511 13614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKC4soy/build/release/bin/kudu as pid 26337
I20260923 15:04:10.859790 13614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKC4soy/build/release/bin/kudu
/tmp/dist-test-taskKC4soy/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.13.75.148:49985/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.75.129:35677
--local_ip_for_outbound_sockets=127.13.75.129
--tserver_master_addrs=127.13.75.190:34053
--webserver_port=42981
--webserver_interface=127.13.75.129
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.129.keytab
--principal=kudu/127.13.75.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.75.148:42213
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.13.75.148:49985/kms with env {KRB5CCNAME=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260923 15:04:10.860136 26406 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dbc551ca623f4882ab0ac309072be9f0: Bootstrap starting.
I20260923 15:04:10.871023 26406 log.cc:824] T 00000000000000000000000000000000 P dbc551ca623f4882ab0ac309072be9f0: Log is configured to *not* fsync() on all Append() calls
I20260923 15:04:10.873533 26406 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dbc551ca623f4882ab0ac309072be9f0: Bootstrap replayed 1/1 log segments. Stats: ops{read=4 overwritten=0 applied=4 ignored=0} inserts{seen=3 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260923 15:04:10.873872 26406 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P dbc551ca623f4882ab0ac309072be9f0: Bootstrap complete.
I20260923 15:04:10.877315 26406 raft_consensus.cc:348] T 00000000000000000000000000000000 P dbc551ca623f4882ab0ac309072be9f0 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbc551ca623f4882ab0ac309072be9f0" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 34053 } }
I20260923 15:04:10.877681 26406 raft_consensus.cc:729] T 00000000000000000000000000000000 P dbc551ca623f4882ab0ac309072be9f0 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: dbc551ca623f4882ab0ac309072be9f0, State: Initialized, Role: FOLLOWER
I20260923 15:04:10.877849 26406 consensus_queue.cc:260] T 00000000000000000000000000000000 P dbc551ca623f4882ab0ac309072be9f0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbc551ca623f4882ab0ac309072be9f0" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 34053 } }
I20260923 15:04:10.877950 26406 raft_consensus.cc:388] T 00000000000000000000000000000000 P dbc551ca623f4882ab0ac309072be9f0 [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 15:04:10.878006 26406 raft_consensus.cc:482] T 00000000000000000000000000000000 P dbc551ca623f4882ab0ac309072be9f0 [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 15:04:10.878047 26406 raft_consensus.cc:3037] T 00000000000000000000000000000000 P dbc551ca623f4882ab0ac309072be9f0 [term 1 FOLLOWER]: Advancing to term 2
I20260923 15:04:10.879554 26406 raft_consensus.cc:504] T 00000000000000000000000000000000 P dbc551ca623f4882ab0ac309072be9f0 [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbc551ca623f4882ab0ac309072be9f0" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 34053 } }
I20260923 15:04:10.879681 26406 leader_election.cc:302] T 00000000000000000000000000000000 P dbc551ca623f4882ab0ac309072be9f0 [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: dbc551ca623f4882ab0ac309072be9f0; no voters:
I20260923 15:04:10.879869 26406 leader_election.cc:288] T 00000000000000000000000000000000 P dbc551ca623f4882ab0ac309072be9f0 [CANDIDATE]: Term 2 election: Requested vote from peers
I20260923 15:04:10.879978 26410 raft_consensus.cc:2781] T 00000000000000000000000000000000 P dbc551ca623f4882ab0ac309072be9f0 [term 2 FOLLOWER]: Leader election won for term 2
I20260923 15:04:10.880198 26406 sys_catalog.cc:564] T 00000000000000000000000000000000 P dbc551ca623f4882ab0ac309072be9f0 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 15:04:10.880228 26410 raft_consensus.cc:686] T 00000000000000000000000000000000 P dbc551ca623f4882ab0ac309072be9f0 [term 2 LEADER]: Becoming Leader. State: Replica: dbc551ca623f4882ab0ac309072be9f0, State: Running, Role: LEADER
I20260923 15:04:10.880513 26410 consensus_queue.cc:237] T 00000000000000000000000000000000 P dbc551ca623f4882ab0ac309072be9f0 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 4, Committed index: 4, Last appended: 1.4, Last appended by leader: 4, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbc551ca623f4882ab0ac309072be9f0" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 34053 } }
I20260923 15:04:10.880887 26411 sys_catalog.cc:455] T 00000000000000000000000000000000 P dbc551ca623f4882ab0ac309072be9f0 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "dbc551ca623f4882ab0ac309072be9f0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbc551ca623f4882ab0ac309072be9f0" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 34053 } } }
I20260923 15:04:10.881088 26411 sys_catalog.cc:458] T 00000000000000000000000000000000 P dbc551ca623f4882ab0ac309072be9f0 [sys.catalog]: This master's current role is: LEADER
I20260923 15:04:10.881515 26419 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 15:04:10.881667 26412 sys_catalog.cc:455] T 00000000000000000000000000000000 P dbc551ca623f4882ab0ac309072be9f0 [sys.catalog]: SysCatalogTable state changed. Reason: New leader dbc551ca623f4882ab0ac309072be9f0. Latest consensus state: current_term: 2 leader_uuid: "dbc551ca623f4882ab0ac309072be9f0" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "dbc551ca623f4882ab0ac309072be9f0" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 34053 } } }
I20260923 15:04:10.881742 26412 sys_catalog.cc:458] T 00000000000000000000000000000000 P dbc551ca623f4882ab0ac309072be9f0 [sys.catalog]: This master's current role is: LEADER
I20260923 15:04:10.881860 26406 ranger_client.cc:318] Using overwritten properties file: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/minicluster-data/master-0/logs/kudu-ranger-subprocess-log4j2.properties
I20260923 15:04:10.882524 26419 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 15:04:10.882682 26419 catalog_manager.cc:1259] Loaded cluster ID: 2414b62f39144bfcbd3161383bbe221b
I20260923 15:04:10.882723 26419 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 15:04:10.886997 26419 catalog_manager.cc:1504] Loading token signing keys...
I20260923 15:04:10.887435 26419 catalog_manager.cc:5879] T 00000000000000000000000000000000 P dbc551ca623f4882ab0ac309072be9f0: Loaded TSK: 0
I20260923 15:04:10.887727 26419 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260923 15:04:10.988221 26408 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:04:10.988597 26408 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:04:10.988638 26408 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:04:10.990543 26408 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:04:10.990653 26408 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:04:10.990715 26408 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.75.129
I20260923 15:04:10.993160 26408 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.75.148:42213
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/minicluster-data/ts-0/wal
--ranger_kms_url=invalid.host:1234/kms,127.13.75.148:49985/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.129.keytab
--principal=kudu/127.13.75.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.75.129:35677
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.75.129
--webserver_port=42981
--tserver_master_addrs=127.13.75.190:34053
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26408
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.75.129
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:04:10.993569 26408 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:04:10.993896 26408 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:04:11.002134 26446 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:04:11.002506 26443 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:04:11.002686 26408 server_base.cc:1034] running on GCE node
W20260923 15:04:11.002913 26444 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:04:11.003203 26408 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:04:11.003530 26408 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:04:11.004748 26408 hybrid_clock.cc:648] HybridClock initialized: now 1790175851004713 us; error 48 us; skew 500 ppm
Sep 23 15:04:11 dist-test-slave-cnb0 krb5kdc[25373](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175851, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:04:11.008304 26408 init.cc:377] Logged in from keytab as kudu/127.13.75.129@KRBTEST.COM (short username kudu)
I20260923 15:04:11.008982 26408 webserver.cc:460] Webserver started at http://127.13.75.129:42981/ using document root <none> and password file <none>
I20260923 15:04:11.009268 26408 fs_manager.cc:359] Metadata directory not provided
I20260923 15:04:11.009361 26408 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 23 15:04:11 dist-test-slave-cnb0 krb5kdc[25373](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175851, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.129@KRBTEST.COM for HTTP/127.13.75.148@KRBTEST.COM
I20260923 15:04:11.042706 26408 fs_manager.cc:709] Time spent opening directory manager: real 0.004s user 0.000s sys 0.001s
I20260923 15:04:11.044595 26454 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:04:11.044780 26408 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.000s
I20260923 15:04:11.044872 26408 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/minicluster-data/ts-0/wal
uuid: "2e9e6fe7c03c4f2680c9679a16e60436"
format_stamp: "Formatted at 2026-09-23 15:04:09 on dist-test-slave-cnb0"
server_key: "2d8f9e57a3fee1a3f3e99b98919f62b8"
server_key_iv: "b1ae33ff9ab1782dc9106c36574a8100"
server_key_version: "kuduclusterkey@0"
I20260923 15:04:11.045372 26408 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:04:11.060604 26408 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:04:11.063129 26408 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:04:11.063301 26408 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:04:11.063586 26408 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 15:04:11.063993 26408 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 15:04:11.064045 26408 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 15:04:11.064079 26408 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 15:04:11.064105 26408 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 15:04:11.071790 26408 rpc_server.cc:307] RPC server started. Bound to: 127.13.75.129:35677
I20260923 15:04:11.071924 26567 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.75.129:35677 every 8 connection(s)
I20260923 15:04:11.072305 26408 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/minicluster-data/ts-0/data/info.pb
I20260923 15:04:11.078639 13614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKC4soy/build/release/bin/kudu as pid 26408
I20260923 15:04:11.078977 13614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKC4soy/build/release/bin/kudu
/tmp/dist-test-taskKC4soy/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.13.75.148:49985/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.75.130:44109
--local_ip_for_outbound_sockets=127.13.75.130
--tserver_master_addrs=127.13.75.190:34053
--webserver_port=42951
--webserver_interface=127.13.75.130
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.130.keytab
--principal=kudu/127.13.75.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.75.148:42213
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.13.75.148:49985/kms with env {KRB5CCNAME=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
Sep 23 15:04:11 dist-test-slave-cnb0 krb5kdc[25373](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175851, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.129@KRBTEST.COM for kudu/127.13.75.190@KRBTEST.COM
I20260923 15:04:11.085417 26571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:11.073289 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.129:35895 (local address 127.13.75.190:34053)
0923 15:04:11.074332 (+ 1043us) server_negotiation.cc:206] Beginning negotiation
0923 15:04:11.074342 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0923 15:04:11.074379 (+ 37us) server_negotiation.cc:407] Connection header received
0923 15:04:11.074540 (+ 161us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:04:11.074546 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:04:11.074606 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:04:11.074736 (+ 130us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:04:11.075579 (+ 843us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:04:11.076524 (+ 945us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:04:11.078001 (+ 1477us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:04:11.078510 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:04:11.078635 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:04:11.081475 (+ 2840us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:04:11.081481 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:04:11.081482 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:04:11.081497 (+ 15us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:04:11.083448 (+ 1951us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:04:11.083935 (+ 487us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:04:11.083939 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:04:11.083939 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:04:11.084041 (+ 102us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:04:11.084326 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:04:11.084327 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:04:11.084327 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:04:11.084591 (+ 264us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:04:11.084684 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0923 15:04:11.085066 (+ 382us) server_negotiation.cc:299] Negotiation successful
0923 15:04:11.085186 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":977,"thread_start_us":73,"threads_started":1}
I20260923 15:04:11.085433 26570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:11.073207 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:34053 (local address 127.13.75.129:35895)
0923 15:04:11.073469 (+ 262us) negotiation.cc:107] Waiting for socket to connect
0923 15:04:11.073514 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0923 15:04:11.074107 (+ 593us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:04:11.074926 (+ 819us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:04:11.074931 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:04:11.075164 (+ 233us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:04:11.075449 (+ 285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:04:11.075456 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:04:11.076636 (+ 1180us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:04:11.076637 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:04:11.077886 (+ 1249us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:04:11.077889 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:04:11.078645 (+ 756us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:04:11.078647 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:04:11.078786 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:04:11.079509 (+ 723us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:04:11.079521 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:04:11.081255 (+ 1734us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:04:11.083570 (+ 2315us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:04:11.083579 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:04:11.083581 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:04:11.083836 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:04:11.084122 (+ 286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:04:11.084123 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:04:11.084124 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:04:11.084249 (+ 125us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:04:11.084677 (+ 428us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:04:11.084683 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:04:11.084868 (+ 185us) client_negotiation.cc:769] Sending connection context
0923 15:04:11.085003 (+ 135us) client_negotiation.cc:240] Negotiation successful
0923 15:04:11.085186 (+ 183us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":221,"thread_start_us":81,"threads_started":1}
I20260923 15:04:11.086584 26568 heartbeater.cc:347] Connected to a master server at 127.13.75.190:34053
I20260923 15:04:11.086824 26568 heartbeater.cc:464] Registering TS with master...
I20260923 15:04:11.087064 26568 heartbeater.cc:511] Master 127.13.75.190:34053 requested a full tablet report, sending...
W20260923 15:04:11.088004 26568 heartbeater.cc:644] Failed to heartbeat to 127.13.75.190:34053 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260923 15:04:11.088068 26568 heartbeater.cc:464] Registering TS with master...
I20260923 15:04:11.088160 26568 heartbeater.cc:511] Master 127.13.75.190:34053 requested a full tablet report, sending...
I20260923 15:04:11.088459 26568 heartbeater.cc:464] Registering TS with master...
I20260923 15:04:11.088557 26568 heartbeater.cc:511] Master 127.13.75.190:34053 requested a full tablet report, sending...
W20260923 15:04:11.088845 26568 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260923 15:04:11.205902 26572 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:04:11.206224 26572 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:04:11.206277 26572 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:04:11.208250 26572 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:04:11.208361 26572 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:04:11.208424 26572 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.75.130
I20260923 15:04:11.210421 26572 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.75.148:42213
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/minicluster-data/ts-1/wal
--ranger_kms_url=invalid.host:1234/kms,127.13.75.148:49985/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.130.keytab
--principal=kudu/127.13.75.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.75.130:44109
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.75.130
--webserver_port=42951
--tserver_master_addrs=127.13.75.190:34053
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26572
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.75.130
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:04:11.210747 26572 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:04:11.211090 26572 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:04:11.217145 26580 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:04:11.217440 26577 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:04:11.217171 26578 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:04:11.217844 26572 server_base.cc:1034] running on GCE node
I20260923 15:04:11.218067 26572 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:04:11.218384 26572 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:04:11.219580 26572 hybrid_clock.cc:648] HybridClock initialized: now 1790175851219547 us; error 46 us; skew 500 ppm
Sep 23 15:04:11 dist-test-slave-cnb0 krb5kdc[25373](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175851, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:04:11.229203 26572 init.cc:377] Logged in from keytab as kudu/127.13.75.130@KRBTEST.COM (short username kudu)
I20260923 15:04:11.229816 26572 webserver.cc:460] Webserver started at http://127.13.75.130:42951/ using document root <none> and password file <none>
I20260923 15:04:11.230131 26572 fs_manager.cc:359] Metadata directory not provided
I20260923 15:04:11.230221 26572 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 23 15:04:11 dist-test-slave-cnb0 krb5kdc[25373](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175851, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.130@KRBTEST.COM for HTTP/127.13.75.148@KRBTEST.COM
I20260923 15:04:11.248814 26572 fs_manager.cc:709] Time spent opening directory manager: real 0.002s user 0.001s sys 0.000s
I20260923 15:04:11.249833 26588 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:04:11.250095 26572 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:04:11.250195 26572 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/minicluster-data/ts-1/wal
uuid: "5d5b6b7baf20426488ded3114f167e34"
format_stamp: "Formatted at 2026-09-23 15:04:10 on dist-test-slave-cnb0"
server_key: "cd0e229d7ecf7560e84311e6abe99573"
server_key_iv: "3fc94cde5e401aef3482d9439917762a"
server_key_version: "kuduclusterkey@0"
I20260923 15:04:11.250633 26572 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:04:11.266499 26572 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:04:11.268903 26572 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:04:11.269080 26572 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:04:11.269363 26572 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 15:04:11.269748 26572 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 15:04:11.269800 26572 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 15:04:11.269840 26572 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 15:04:11.269863 26572 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 15:04:11.277055 26572 rpc_server.cc:307] RPC server started. Bound to: 127.13.75.130:44109
I20260923 15:04:11.277467 26572 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/minicluster-data/ts-1/data/info.pb
I20260923 15:04:11.278142 13614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKC4soy/build/release/bin/kudu as pid 26572
I20260923 15:04:11.278443 13614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKC4soy/build/release/bin/kudu
/tmp/dist-test-taskKC4soy/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--encryption_key_provider=ranger-kms
--encryption_cluster_key_name=kuduclusterkey
--ranger_kms_url=127.13.75.148:49985/kms
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.75.131:39955
--local_ip_for_outbound_sockets=127.13.75.131
--tserver_master_addrs=127.13.75.190:34053
--webserver_port=46697
--webserver_interface=127.13.75.131
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.131.keytab
--principal=kudu/127.13.75.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.75.148:42213
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--ranger_kms_url=invalid.host:1234/kms,127.13.75.148:49985/kms with env {KRB5CCNAME=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260923 15:04:11.278467 26701 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.75.130:44109 every 8 connection(s)
Sep 23 15:04:11 dist-test-slave-cnb0 krb5kdc[25373](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175851, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.130@KRBTEST.COM for kudu/127.13.75.190@KRBTEST.COM
I20260923 15:04:11.294982 26571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:11.280692 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.130:49201 (local address 127.13.75.190:34053)
0923 15:04:11.280751 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0923 15:04:11.280753 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:04:11.280769 (+ 16us) server_negotiation.cc:407] Connection header received
0923 15:04:11.281553 (+ 784us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:04:11.281555 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:04:11.281569 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:04:11.281603 (+ 34us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:04:11.282474 (+ 871us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:04:11.283298 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:04:11.284897 (+ 1599us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:04:11.285194 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:04:11.285244 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:04:11.291204 (+ 5960us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:04:11.291206 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:04:11.291207 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:04:11.291216 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:04:11.293051 (+ 1835us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:04:11.293681 (+ 630us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:04:11.293683 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:04:11.293683 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:04:11.293763 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:04:11.294132 (+ 369us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:04:11.294133 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:04:11.294133 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:04:11.294292 (+ 159us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:04:11.294364 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0923 15:04:11.294749 (+ 385us) server_negotiation.cc:299] Negotiation successful
0923 15:04:11.294844 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260923 15:04:11.295187 26705 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:11.280426 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:34053 (local address 127.13.75.130:49201)
0923 15:04:11.280602 (+ 176us) negotiation.cc:107] Waiting for socket to connect
0923 15:04:11.280647 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0923 15:04:11.281415 (+ 768us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:04:11.281802 (+ 387us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:04:11.281808 (+ 6us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:04:11.282081 (+ 273us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:04:11.282366 (+ 285us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:04:11.282373 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:04:11.283421 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:04:11.283422 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:04:11.284768 (+ 1346us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:04:11.284772 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:04:11.285368 (+ 596us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:04:11.285369 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:04:11.285525 (+ 156us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:04:11.286181 (+ 656us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:04:11.286194 (+ 13us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:04:11.291044 (+ 4850us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:04:11.293199 (+ 2155us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:04:11.293211 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:04:11.293214 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:04:11.293583 (+ 369us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:04:11.293858 (+ 275us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:04:11.293859 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:04:11.293859 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:04:11.294026 (+ 167us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:04:11.294378 (+ 352us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:04:11.294385 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:04:11.294579 (+ 194us) client_negotiation.cc:769] Sending connection context
0923 15:04:11.294721 (+ 142us) client_negotiation.cc:240] Negotiation successful
0923 15:04:11.294931 (+ 210us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":98,"threads_started":1}
I20260923 15:04:11.295704 26702 heartbeater.cc:347] Connected to a master server at 127.13.75.190:34053
I20260923 15:04:11.295809 26702 heartbeater.cc:464] Registering TS with master...
I20260923 15:04:11.296033 26702 heartbeater.cc:511] Master 127.13.75.190:34053 requested a full tablet report, sending...
W20260923 15:04:11.296680 26702 heartbeater.cc:644] Failed to heartbeat to 127.13.75.190:34053 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260923 15:04:11.296747 26702 heartbeater.cc:464] Registering TS with master...
I20260923 15:04:11.296808 26702 heartbeater.cc:511] Master 127.13.75.190:34053 requested a full tablet report, sending...
I20260923 15:04:11.297102 26702 heartbeater.cc:464] Registering TS with master...
I20260923 15:04:11.297168 26702 heartbeater.cc:511] Master 127.13.75.190:34053 requested a full tablet report, sending...
W20260923 15:04:11.297453 26702 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260923 15:04:11.389313 26704 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:04:11.389627 26704 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:04:11.389673 26704 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:04:11.391677 26704 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:04:11.391736 26704 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:04:11.391780 26704 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.75.131
I20260923 15:04:11.394204 26704 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.75.148:42213
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--encryption_cluster_key_name=kuduclusterkey
--encryption_key_provider=ranger-kms
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/minicluster-data/ts-2/wal
--ranger_kms_url=invalid.host:1234/kms,127.13.75.148:49985/kms
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.131.keytab
--principal=kudu/127.13.75.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.75.131:39955
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.75.131
--webserver_port=46697
--tserver_master_addrs=127.13.75.190:34053
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26704
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.75.131
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:04:11.394534 26704 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:04:11.394853 26704 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:04:11.402230 26711 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:04:11.402241 26710 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:04:11.402696 26713 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:04:11.402982 26704 server_base.cc:1034] running on GCE node
I20260923 15:04:11.403203 26704 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:04:11.403496 26704 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:04:11.404675 26704 hybrid_clock.cc:648] HybridClock initialized: now 1790175851404639 us; error 48 us; skew 500 ppm
Sep 23 15:04:11 dist-test-slave-cnb0 krb5kdc[25373](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175851, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:04:11.408280 26704 init.cc:377] Logged in from keytab as kudu/127.13.75.131@KRBTEST.COM (short username kudu)
I20260923 15:04:11.408872 26704 webserver.cc:460] Webserver started at http://127.13.75.131:46697/ using document root <none> and password file <none>
I20260923 15:04:11.409168 26704 fs_manager.cc:359] Metadata directory not provided
I20260923 15:04:11.409260 26704 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
Sep 23 15:04:11 dist-test-slave-cnb0 krb5kdc[25373](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175851, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.131@KRBTEST.COM for HTTP/127.13.75.148@KRBTEST.COM
I20260923 15:04:11.432821 26704 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:04:11.435418 26722 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:04:11.435720 26704 fs_manager.cc:725] Time spent opening block manager: real 0.002s user 0.000s sys 0.000s
I20260923 15:04:11.435838 26704 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/minicluster-data/ts-2/wal
uuid: "2301dc7034fb46b8bb6165fd0e620ba2"
format_stamp: "Formatted at 2026-09-23 15:04:10 on dist-test-slave-cnb0"
server_key: "7b9576c7abc15debf9e99b7994dee75b"
server_key_iv: "2d0a878779252fda70a6ad68f362d6fd"
server_key_version: "kuduclusterkey@0"
I20260923 15:04:11.436388 26704 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:04:11.462845 26704 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:04:11.465446 26704 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:04:11.465618 26704 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:04:11.465929 26704 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 15:04:11.466367 26704 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 15:04:11.466428 26704 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 15:04:11.466471 26704 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 15:04:11.466498 26704 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 15:04:11.475672 26704 rpc_server.cc:307] RPC server started. Bound to: 127.13.75.131:39955
I20260923 15:04:11.475749 26835 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.75.131:39955 every 8 connection(s)
I20260923 15:04:11.476109 26704 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/minicluster-data/ts-2/data/info.pb
Sep 23 15:04:11 dist-test-slave-cnb0 krb5kdc[25373](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175851, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.131@KRBTEST.COM for kudu/127.13.75.190@KRBTEST.COM
I20260923 15:04:11.484726 13614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKC4soy/build/release/bin/kudu as pid 26704
I20260923 15:04:11.487668 26571 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:11.476934 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.131:38759 (local address 127.13.75.190:34053)
0923 15:04:11.477002 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0923 15:04:11.477004 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:04:11.477342 (+ 338us) server_negotiation.cc:407] Connection header received
0923 15:04:11.477960 (+ 618us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:04:11.477962 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:04:11.477975 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:04:11.478037 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:04:11.478655 (+ 618us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:04:11.479297 (+ 642us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:04:11.480993 (+ 1696us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:04:11.481325 (+ 332us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:04:11.481375 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:04:11.484014 (+ 2639us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:04:11.484016 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:04:11.484016 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:04:11.484025 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:04:11.485839 (+ 1814us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:04:11.486376 (+ 537us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:04:11.486377 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:04:11.486377 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:04:11.486454 (+ 77us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:04:11.486789 (+ 335us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:04:11.486790 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:04:11.486791 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:04:11.486978 (+ 187us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:04:11.487031 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0923 15:04:11.487478 (+ 447us) server_negotiation.cc:299] Negotiation successful
0923 15:04:11.487564 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260923 15:04:11.488826 26838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:11.476991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:34053 (local address 127.13.75.131:38759)
0923 15:04:11.477182 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0923 15:04:11.477229 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0923 15:04:11.477842 (+ 613us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:04:11.478116 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:04:11.478121 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:04:11.478323 (+ 202us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:04:11.478541 (+ 218us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:04:11.478545 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:04:11.479408 (+ 863us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:04:11.479409 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:04:11.480881 (+ 1472us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:04:11.480884 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:04:11.481524 (+ 640us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:04:11.481525 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:04:11.481677 (+ 152us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:04:11.482273 (+ 596us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:04:11.482284 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:04:11.483887 (+ 1603us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:04:11.485929 (+ 2042us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:04:11.485936 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:04:11.485939 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:04:11.486294 (+ 355us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:04:11.486538 (+ 244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:04:11.486539 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:04:11.486539 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:04:11.486698 (+ 159us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:04:11.487090 (+ 392us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:04:11.487097 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:04:11.487288 (+ 191us) client_negotiation.cc:769] Sending connection context
0923 15:04:11.487451 (+ 163us) client_negotiation.cc:240] Negotiation successful
0923 15:04:11.487620 (+ 169us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":114,"threads_started":1}
I20260923 15:04:11.489537 26836 heartbeater.cc:347] Connected to a master server at 127.13.75.190:34053
I20260923 15:04:11.489639 26836 heartbeater.cc:464] Registering TS with master...
I20260923 15:04:11.489836 26836 heartbeater.cc:511] Master 127.13.75.190:34053 requested a full tablet report, sending...
W20260923 15:04:11.490482 26836 heartbeater.cc:644] Failed to heartbeat to 127.13.75.190:34053 (0 consecutive failures): Service unavailable: Catalog manager is not initialized. State: Starting
I20260923 15:04:11.490576 26836 heartbeater.cc:464] Registering TS with master...
I20260923 15:04:11.490656 26836 heartbeater.cc:511] Master 127.13.75.190:34053 requested a full tablet report, sending...
I20260923 15:04:11.491118 26836 heartbeater.cc:464] Registering TS with master...
I20260923 15:04:11.491214 26836 heartbeater.cc:511] Master 127.13.75.190:34053 requested a full tablet report, sending...
W20260923 15:04:11.491573 26836 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260923 15:04:11.496114 26840 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:11.486007 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:32856 (local address 127.13.75.190:34053)
0923 15:04:11.486111 (+ 104us) server_negotiation.cc:206] Beginning negotiation
0923 15:04:11.486113 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:04:11.486120 (+ 7us) server_negotiation.cc:407] Connection header received
0923 15:04:11.486143 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:04:11.486143 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:04:11.486157 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:04:11.486231 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:04:11.487990 (+ 1759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:04:11.488518 (+ 528us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:04:11.490086 (+ 1568us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:04:11.490914 (+ 828us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:04:11.490998 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:04:11.492622 (+ 1624us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:04:11.492624 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:04:11.492624 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:04:11.492632 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:04:11.494460 (+ 1828us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:04:11.494981 (+ 521us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:04:11.494982 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:04:11.494983 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:04:11.495053 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:04:11.495349 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:04:11.495350 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:04:11.495350 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:04:11.495529 (+ 179us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:04:11.495717 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0923 15:04:11.495897 (+ 180us) server_negotiation.cc:299] Negotiation successful
0923 15:04:11.496002 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86,"thread_start_us":54,"threads_started":1}
Sep 23 15:04:12 dist-test-slave-cnb0 krb5kdc[25373](info): AS_REQ (1 etypes {17}) 127.0.0.1: ISSUE: authtime 1790175852, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:04:12.089043 26568 heartbeater.cc:464] Registering TS with master...
I20260923 15:04:12.089210 26568 heartbeater.cc:511] Master 127.13.75.190:34053 requested a full tablet report, sending...
I20260923 15:04:12.089622 26370 ts_manager.cc:194] Registered new tserver with Master: 2e9e6fe7c03c4f2680c9679a16e60436 (127.13.75.129:35677)
I20260923 15:04:12.090615 26370 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.75.129@KRBTEST.COM'} at 127.13.75.129:35895
I20260923 15:04:12.297629 26702 heartbeater.cc:464] Registering TS with master...
I20260923 15:04:12.297787 26702 heartbeater.cc:511] Master 127.13.75.190:34053 requested a full tablet report, sending...
I20260923 15:04:12.298040 26370 ts_manager.cc:194] Registered new tserver with Master: 5d5b6b7baf20426488ded3114f167e34 (127.13.75.130:44109)
I20260923 15:04:12.298556 26370 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.75.130@KRBTEST.COM'} at 127.13.75.130:49201
Sep 23 15:04:12 dist-test-slave-cnb0 krb5kdc[25373](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175852, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.190@KRBTEST.COM for HTTP/127.13.75.148@KRBTEST.COM
I20260923 15:04:12.491784 26836 heartbeater.cc:464] Registering TS with master...
I20260923 15:04:12.491995 26836 heartbeater.cc:511] Master 127.13.75.190:34053 requested a full tablet report, sending...
I20260923 15:04:12.492403 26370 ts_manager.cc:194] Registered new tserver with Master: 2301dc7034fb46b8bb6165fd0e620ba2 (127.13.75.131:39955)
I20260923 15:04:12.492805 26370 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.75.131@KRBTEST.COM'} at 127.13.75.131:38759
I20260923 15:04:12.495479 13614 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260923 15:04:12.504387 26858 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:12.496470 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38616 (local address 127.13.75.190:34053)
0923 15:04:12.496709 (+ 239us) server_negotiation.cc:206] Beginning negotiation
0923 15:04:12.496711 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:04:12.496718 (+ 7us) server_negotiation.cc:407] Connection header received
0923 15:04:12.496742 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:04:12.496743 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:04:12.496755 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:04:12.496821 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:04:12.497543 (+ 722us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:04:12.498257 (+ 714us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:04:12.499357 (+ 1100us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:04:12.499654 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:04:12.499730 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:04:12.501222 (+ 1492us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:04:12.501223 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:04:12.501223 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:04:12.501231 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:04:12.502985 (+ 1754us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:04:12.503428 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:04:12.503429 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:04:12.503429 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:04:12.503495 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:04:12.503707 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:04:12.503708 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:04:12.503709 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:04:12.503844 (+ 135us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:04:12.503944 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0923 15:04:12.504087 (+ 143us) server_negotiation.cc:299] Negotiation successful
0923 15:04:12.504176 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":219,"thread_start_us":69,"threads_started":1}
I20260923 15:04:12.506597 26370 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38616:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 15:04:12.508976 26370 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 15:04:12.523908 26868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:12.518847 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45478 (local address 127.13.75.130:44109)
0923 15:04:12.519179 (+ 332us) server_negotiation.cc:206] Beginning negotiation
0923 15:04:12.519183 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 15:04:12.519406 (+ 223us) server_negotiation.cc:407] Connection header received
0923 15:04:12.519505 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:04:12.519508 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:04:12.519524 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:04:12.519576 (+ 52us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:04:12.519796 (+ 220us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:04:12.520843 (+ 1047us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:04:12.522548 (+ 1705us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:04:12.523377 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:04:12.523429 (+ 52us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:04:12.523487 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0923 15:04:12.523747 (+ 260us) server_negotiation.cc:299] Negotiation successful
0923 15:04:12.523791 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":302,"thread_start_us":67,"threads_started":1}
I20260923 15:04:12.523989 26863 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:12.518402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.131:39955 (local address 127.0.0.1:38956)
0923 15:04:12.518841 (+ 439us) negotiation.cc:107] Waiting for socket to connect
0923 15:04:12.518930 (+ 89us) client_negotiation.cc:174] Beginning negotiation
0923 15:04:12.519044 (+ 114us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:04:12.519461 (+ 417us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:04:12.519464 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:04:12.519467 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:04:12.519543 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:04:12.519545 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:04:12.521125 (+ 1580us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:04:12.521126 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:04:12.522438 (+ 1312us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:04:12.522441 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:04:12.523747 (+ 1306us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:04:12.523748 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:04:12.523812 (+ 64us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:04:12.523817 (+ 5us) client_negotiation.cc:769] Sending connection context
0923 15:04:12.523844 (+ 27us) client_negotiation.cc:240] Negotiation successful
0923 15:04:12.523905 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":404,"thread_start_us":96,"threads_started":1}
I20260923 15:04:12.524156 26864 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:12.518755 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.130:44109 (local address 127.0.0.1:45478)
0923 15:04:12.519358 (+ 603us) negotiation.cc:107] Waiting for socket to connect
0923 15:04:12.519366 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0923 15:04:12.519430 (+ 64us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:04:12.519671 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:04:12.519672 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:04:12.519674 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:04:12.519726 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:04:12.519727 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:04:12.520959 (+ 1232us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:04:12.520960 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:04:12.522444 (+ 1484us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:04:12.522447 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:04:12.523548 (+ 1101us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:04:12.523549 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:04:12.523630 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:04:12.523643 (+ 13us) client_negotiation.cc:769] Sending connection context
0923 15:04:12.524018 (+ 375us) client_negotiation.cc:240] Negotiation successful
0923 15:04:12.524070 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":590,"mutex_wait_us":227,"thread_start_us":70,"threads_started":1}
I20260923 15:04:12.524801 26865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:12.518664 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38956 (local address 127.13.75.131:39955)
0923 15:04:12.519273 (+ 609us) server_negotiation.cc:206] Beginning negotiation
0923 15:04:12.519278 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 15:04:12.519288 (+ 10us) server_negotiation.cc:407] Connection header received
0923 15:04:12.519324 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:04:12.519326 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:04:12.519343 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:04:12.519616 (+ 273us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:04:12.519691 (+ 75us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:04:12.520758 (+ 1067us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:04:12.522542 (+ 1784us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:04:12.523652 (+ 1110us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:04:12.524593 (+ 941us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:04:12.524659 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0923 15:04:12.524685 (+ 26us) server_negotiation.cc:299] Negotiation successful
0923 15:04:12.524725 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":578,"thread_start_us":57,"threads_started":1}
I20260923 15:04:12.524884 26636 tablet_service.cc:1467] Processing CreateTablet for tablet 260cf94d3706497ea8436fac38ac3c45 (DEFAULT_TABLE table=test-table [id=be430e9abbfa468493753f1d1ec40769]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 15:04:12.525259 26636 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 260cf94d3706497ea8436fac38ac3c45. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:04:12.525601 26770 tablet_service.cc:1467] Processing CreateTablet for tablet 260cf94d3706497ea8436fac38ac3c45 (DEFAULT_TABLE table=test-table [id=be430e9abbfa468493753f1d1ec40769]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 15:04:12.525945 26770 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 260cf94d3706497ea8436fac38ac3c45. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:04:12.529321 26869 tablet_bootstrap.cc:492] T 260cf94d3706497ea8436fac38ac3c45 P 5d5b6b7baf20426488ded3114f167e34: Bootstrap starting.
I20260923 15:04:12.530016 26869 tablet_bootstrap.cc:654] T 260cf94d3706497ea8436fac38ac3c45 P 5d5b6b7baf20426488ded3114f167e34: Neither blocks nor log segments found. Creating new log.
I20260923 15:04:12.530344 26869 log.cc:824] T 260cf94d3706497ea8436fac38ac3c45 P 5d5b6b7baf20426488ded3114f167e34: Log is configured to *not* fsync() on all Append() calls
I20260923 15:04:12.530664 26870 tablet_bootstrap.cc:492] T 260cf94d3706497ea8436fac38ac3c45 P 2301dc7034fb46b8bb6165fd0e620ba2: Bootstrap starting.
I20260923 15:04:12.531353 26870 tablet_bootstrap.cc:654] T 260cf94d3706497ea8436fac38ac3c45 P 2301dc7034fb46b8bb6165fd0e620ba2: Neither blocks nor log segments found. Creating new log.
I20260923 15:04:12.531664 26870 log.cc:824] T 260cf94d3706497ea8436fac38ac3c45 P 2301dc7034fb46b8bb6165fd0e620ba2: Log is configured to *not* fsync() on all Append() calls
I20260923 15:04:12.532425 26869 tablet_bootstrap.cc:492] T 260cf94d3706497ea8436fac38ac3c45 P 5d5b6b7baf20426488ded3114f167e34: No bootstrap required, opened a new log
I20260923 15:04:12.532505 26869 ts_tablet_manager.cc:1397] T 260cf94d3706497ea8436fac38ac3c45 P 5d5b6b7baf20426488ded3114f167e34: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260923 15:04:12.533679 26867 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:12.518402 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.129:35677 (local address 127.0.0.1:45320)
0923 15:04:12.519229 (+ 827us) negotiation.cc:107] Waiting for socket to connect
0923 15:04:12.519238 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 15:04:12.519276 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:04:12.524360 (+ 5084us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:04:12.524361 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:04:12.524363 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:04:12.524403 (+ 40us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:04:12.524404 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:04:12.531087 (+ 6683us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:04:12.531088 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:04:12.532301 (+ 1213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:04:12.532305 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:04:12.533425 (+ 1120us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:04:12.533426 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:04:12.533506 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:04:12.533512 (+ 6us) client_negotiation.cc:769] Sending connection context
0923 15:04:12.533548 (+ 36us) client_negotiation.cc:240] Negotiation successful
0923 15:04:12.533596 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":810,"mutex_wait_us":2,"thread_start_us":87,"threads_started":1}
I20260923 15:04:12.534085 26866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:12.518663 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45320 (local address 127.13.75.129:35677)
0923 15:04:12.521303 (+ 2640us) server_negotiation.cc:206] Beginning negotiation
0923 15:04:12.521309 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 15:04:12.521320 (+ 11us) server_negotiation.cc:407] Connection header received
0923 15:04:12.521355 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:04:12.521357 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:04:12.521372 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:04:12.524484 (+ 3112us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:04:12.524569 (+ 85us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:04:12.530952 (+ 6383us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:04:12.532422 (+ 1470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:04:12.533316 (+ 894us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:04:12.533845 (+ 529us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:04:12.533933 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0923 15:04:12.533962 (+ 29us) server_negotiation.cc:299] Negotiation successful
0923 15:04:12.534005 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2602,"thread_start_us":53,"threads_started":1}
I20260923 15:04:12.534760 26484 tablet_service.cc:1467] Processing CreateTablet for tablet 260cf94d3706497ea8436fac38ac3c45 (DEFAULT_TABLE table=test-table [id=be430e9abbfa468493753f1d1ec40769]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 15:04:12.535146 26484 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 260cf94d3706497ea8436fac38ac3c45. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:04:12.535369 26870 tablet_bootstrap.cc:492] T 260cf94d3706497ea8436fac38ac3c45 P 2301dc7034fb46b8bb6165fd0e620ba2: No bootstrap required, opened a new log
I20260923 15:04:12.535436 26870 ts_tablet_manager.cc:1397] T 260cf94d3706497ea8436fac38ac3c45 P 2301dc7034fb46b8bb6165fd0e620ba2: Time spent bootstrapping tablet: real 0.005s user 0.001s sys 0.000s
I20260923 15:04:12.535596 26869 raft_consensus.cc:348] T 260cf94d3706497ea8436fac38ac3c45 P 5d5b6b7baf20426488ded3114f167e34 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2301dc7034fb46b8bb6165fd0e620ba2" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 39955 } } peers { permanent_uuid: "5d5b6b7baf20426488ded3114f167e34" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 44109 } } peers { permanent_uuid: "2e9e6fe7c03c4f2680c9679a16e60436" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 35677 } }
I20260923 15:04:12.535763 26869 raft_consensus.cc:374] T 260cf94d3706497ea8436fac38ac3c45 P 5d5b6b7baf20426488ded3114f167e34 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:04:12.535794 26869 raft_consensus.cc:729] T 260cf94d3706497ea8436fac38ac3c45 P 5d5b6b7baf20426488ded3114f167e34 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5d5b6b7baf20426488ded3114f167e34, State: Initialized, Role: FOLLOWER
I20260923 15:04:12.535962 26869 consensus_queue.cc:260] T 260cf94d3706497ea8436fac38ac3c45 P 5d5b6b7baf20426488ded3114f167e34 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2301dc7034fb46b8bb6165fd0e620ba2" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 39955 } } peers { permanent_uuid: "5d5b6b7baf20426488ded3114f167e34" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 44109 } } peers { permanent_uuid: "2e9e6fe7c03c4f2680c9679a16e60436" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 35677 } }
I20260923 15:04:12.536278 26869 ts_tablet_manager.cc:1428] T 260cf94d3706497ea8436fac38ac3c45 P 5d5b6b7baf20426488ded3114f167e34: Time spent starting tablet: real 0.004s user 0.000s sys 0.003s
I20260923 15:04:12.536401 26702 heartbeater.cc:503] Master 127.13.75.190:34053 was elected leader, sending a full tablet report...
I20260923 15:04:12.538098 26870 raft_consensus.cc:348] T 260cf94d3706497ea8436fac38ac3c45 P 2301dc7034fb46b8bb6165fd0e620ba2 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2301dc7034fb46b8bb6165fd0e620ba2" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 39955 } } peers { permanent_uuid: "5d5b6b7baf20426488ded3114f167e34" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 44109 } } peers { permanent_uuid: "2e9e6fe7c03c4f2680c9679a16e60436" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 35677 } }
I20260923 15:04:12.538223 26870 raft_consensus.cc:374] T 260cf94d3706497ea8436fac38ac3c45 P 2301dc7034fb46b8bb6165fd0e620ba2 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:04:12.538244 26870 raft_consensus.cc:729] T 260cf94d3706497ea8436fac38ac3c45 P 2301dc7034fb46b8bb6165fd0e620ba2 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2301dc7034fb46b8bb6165fd0e620ba2, State: Initialized, Role: FOLLOWER
I20260923 15:04:12.538373 26870 consensus_queue.cc:260] T 260cf94d3706497ea8436fac38ac3c45 P 2301dc7034fb46b8bb6165fd0e620ba2 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2301dc7034fb46b8bb6165fd0e620ba2" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 39955 } } peers { permanent_uuid: "5d5b6b7baf20426488ded3114f167e34" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 44109 } } peers { permanent_uuid: "2e9e6fe7c03c4f2680c9679a16e60436" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 35677 } }
I20260923 15:04:12.538668 26870 ts_tablet_manager.cc:1428] T 260cf94d3706497ea8436fac38ac3c45 P 2301dc7034fb46b8bb6165fd0e620ba2: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260923 15:04:12.538681 26836 heartbeater.cc:503] Master 127.13.75.190:34053 was elected leader, sending a full tablet report...
I20260923 15:04:12.539269 26875 tablet_bootstrap.cc:492] T 260cf94d3706497ea8436fac38ac3c45 P 2e9e6fe7c03c4f2680c9679a16e60436: Bootstrap starting.
I20260923 15:04:12.540071 26875 tablet_bootstrap.cc:654] T 260cf94d3706497ea8436fac38ac3c45 P 2e9e6fe7c03c4f2680c9679a16e60436: Neither blocks nor log segments found. Creating new log.
I20260923 15:04:12.540472 26875 log.cc:824] T 260cf94d3706497ea8436fac38ac3c45 P 2e9e6fe7c03c4f2680c9679a16e60436: Log is configured to *not* fsync() on all Append() calls
I20260923 15:04:12.541311 26875 tablet_bootstrap.cc:492] T 260cf94d3706497ea8436fac38ac3c45 P 2e9e6fe7c03c4f2680c9679a16e60436: No bootstrap required, opened a new log
I20260923 15:04:12.541394 26875 ts_tablet_manager.cc:1397] T 260cf94d3706497ea8436fac38ac3c45 P 2e9e6fe7c03c4f2680c9679a16e60436: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 15:04:12.543648 26875 raft_consensus.cc:348] T 260cf94d3706497ea8436fac38ac3c45 P 2e9e6fe7c03c4f2680c9679a16e60436 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2301dc7034fb46b8bb6165fd0e620ba2" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 39955 } } peers { permanent_uuid: "5d5b6b7baf20426488ded3114f167e34" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 44109 } } peers { permanent_uuid: "2e9e6fe7c03c4f2680c9679a16e60436" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 35677 } }
I20260923 15:04:12.543807 26875 raft_consensus.cc:374] T 260cf94d3706497ea8436fac38ac3c45 P 2e9e6fe7c03c4f2680c9679a16e60436 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:04:12.543849 26875 raft_consensus.cc:729] T 260cf94d3706497ea8436fac38ac3c45 P 2e9e6fe7c03c4f2680c9679a16e60436 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2e9e6fe7c03c4f2680c9679a16e60436, State: Initialized, Role: FOLLOWER
I20260923 15:04:12.543985 26875 consensus_queue.cc:260] T 260cf94d3706497ea8436fac38ac3c45 P 2e9e6fe7c03c4f2680c9679a16e60436 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2301dc7034fb46b8bb6165fd0e620ba2" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 39955 } } peers { permanent_uuid: "5d5b6b7baf20426488ded3114f167e34" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 44109 } } peers { permanent_uuid: "2e9e6fe7c03c4f2680c9679a16e60436" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 35677 } }
I20260923 15:04:12.544281 26568 heartbeater.cc:503] Master 127.13.75.190:34053 was elected leader, sending a full tablet report...
I20260923 15:04:12.544279 26875 ts_tablet_manager.cc:1428] T 260cf94d3706497ea8436fac38ac3c45 P 2e9e6fe7c03c4f2680c9679a16e60436: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
W20260923 15:04:12.573681 26569 tablet.cc:2369] T 260cf94d3706497ea8436fac38ac3c45 P 2e9e6fe7c03c4f2680c9679a16e60436: Can't schedule compaction. Clean time has not been advanced past its initial value.
Sep 23 15:04:12 dist-test-slave-cnb0 krb5kdc[25373](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175852, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.190@KRBTEST.COM for HTTP/127.13.75.148@KRBTEST.COM
W20260923 15:04:12.727629 26837 tablet.cc:2369] T 260cf94d3706497ea8436fac38ac3c45 P 2301dc7034fb46b8bb6165fd0e620ba2: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 15:04:12.748062 26877 raft_consensus.cc:482] T 260cf94d3706497ea8436fac38ac3c45 P 2e9e6fe7c03c4f2680c9679a16e60436 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 15:04:12.748171 26877 raft_consensus.cc:504] T 260cf94d3706497ea8436fac38ac3c45 P 2e9e6fe7c03c4f2680c9679a16e60436 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2301dc7034fb46b8bb6165fd0e620ba2" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 39955 } } peers { permanent_uuid: "5d5b6b7baf20426488ded3114f167e34" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 44109 } } peers { permanent_uuid: "2e9e6fe7c03c4f2680c9679a16e60436" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 35677 } }
I20260923 15:04:12.748557 26877 leader_election.cc:288] T 260cf94d3706497ea8436fac38ac3c45 P 2e9e6fe7c03c4f2680c9679a16e60436 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 2301dc7034fb46b8bb6165fd0e620ba2 (127.13.75.131:39955), 5d5b6b7baf20426488ded3114f167e34 (127.13.75.130:44109)
I20260923 15:04:12.752319 26868 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:12.748929 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.129:45545 (local address 127.13.75.130:44109)
0923 15:04:12.748979 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0923 15:04:12.748982 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:04:12.749203 (+ 221us) server_negotiation.cc:407] Connection header received
0923 15:04:12.749262 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:04:12.749263 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:04:12.749273 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:04:12.749321 (+ 48us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:04:12.749565 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:04:12.750286 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:04:12.751561 (+ 1275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:04:12.751949 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:04:12.752034 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:04:12.752046 (+ 12us) server_negotiation.cc:1035] Waiting for connection context
0923 15:04:12.752203 (+ 157us) server_negotiation.cc:299] Negotiation successful
0923 15:04:12.752242 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260923 15:04:12.752345 26881 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:12.748699 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.130:44109 (local address 127.13.75.129:45545)
0923 15:04:12.749144 (+ 445us) negotiation.cc:107] Waiting for socket to connect
0923 15:04:12.749151 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0923 15:04:12.749207 (+ 56us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:04:12.749374 (+ 167us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:04:12.749375 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:04:12.749377 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:04:12.749476 (+ 99us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:04:12.749479 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:04:12.750453 (+ 974us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:04:12.750454 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:04:12.751441 (+ 987us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:04:12.751444 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:04:12.752052 (+ 608us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:04:12.752054 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:04:12.752122 (+ 68us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:04:12.752125 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 15:04:12.752195 (+ 70us) client_negotiation.cc:240] Negotiation successful
0923 15:04:12.752238 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":436,"spinlock_wait_cycles":13056,"thread_start_us":107,"threads_started":1}
I20260923 15:04:12.752763 26655 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "260cf94d3706497ea8436fac38ac3c45" candidate_uuid: "2e9e6fe7c03c4f2680c9679a16e60436" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d5b6b7baf20426488ded3114f167e34" is_pre_election: true
I20260923 15:04:12.752904 26655 raft_consensus.cc:2454] T 260cf94d3706497ea8436fac38ac3c45 P 5d5b6b7baf20426488ded3114f167e34 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2e9e6fe7c03c4f2680c9679a16e60436 in term 0.
I20260923 15:04:12.753129 26456 leader_election.cc:302] T 260cf94d3706497ea8436fac38ac3c45 P 2e9e6fe7c03c4f2680c9679a16e60436 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2e9e6fe7c03c4f2680c9679a16e60436, 5d5b6b7baf20426488ded3114f167e34; no voters:
I20260923 15:04:12.753259 26877 raft_consensus.cc:2781] T 260cf94d3706497ea8436fac38ac3c45 P 2e9e6fe7c03c4f2680c9679a16e60436 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 15:04:12.753314 26877 raft_consensus.cc:482] T 260cf94d3706497ea8436fac38ac3c45 P 2e9e6fe7c03c4f2680c9679a16e60436 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 15:04:12.753337 26877 raft_consensus.cc:3037] T 260cf94d3706497ea8436fac38ac3c45 P 2e9e6fe7c03c4f2680c9679a16e60436 [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:04:12.754145 26877 raft_consensus.cc:504] T 260cf94d3706497ea8436fac38ac3c45 P 2e9e6fe7c03c4f2680c9679a16e60436 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2301dc7034fb46b8bb6165fd0e620ba2" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 39955 } } peers { permanent_uuid: "5d5b6b7baf20426488ded3114f167e34" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 44109 } } peers { permanent_uuid: "2e9e6fe7c03c4f2680c9679a16e60436" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 35677 } }
I20260923 15:04:12.754313 26877 leader_election.cc:288] T 260cf94d3706497ea8436fac38ac3c45 P 2e9e6fe7c03c4f2680c9679a16e60436 [CANDIDATE]: Term 1 election: Requested vote from peers 2301dc7034fb46b8bb6165fd0e620ba2 (127.13.75.131:39955), 5d5b6b7baf20426488ded3114f167e34 (127.13.75.130:44109)
I20260923 15:04:12.754501 26655 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "260cf94d3706497ea8436fac38ac3c45" candidate_uuid: "2e9e6fe7c03c4f2680c9679a16e60436" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5d5b6b7baf20426488ded3114f167e34"
I20260923 15:04:12.754585 26655 raft_consensus.cc:3037] T 260cf94d3706497ea8436fac38ac3c45 P 5d5b6b7baf20426488ded3114f167e34 [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:04:12.755550 26655 raft_consensus.cc:2454] T 260cf94d3706497ea8436fac38ac3c45 P 5d5b6b7baf20426488ded3114f167e34 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2e9e6fe7c03c4f2680c9679a16e60436 in term 1.
I20260923 15:04:12.755739 26456 leader_election.cc:302] T 260cf94d3706497ea8436fac38ac3c45 P 2e9e6fe7c03c4f2680c9679a16e60436 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2e9e6fe7c03c4f2680c9679a16e60436, 5d5b6b7baf20426488ded3114f167e34; no voters:
I20260923 15:04:12.755858 26877 raft_consensus.cc:2781] T 260cf94d3706497ea8436fac38ac3c45 P 2e9e6fe7c03c4f2680c9679a16e60436 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 15:04:12.756062 26877 raft_consensus.cc:686] T 260cf94d3706497ea8436fac38ac3c45 P 2e9e6fe7c03c4f2680c9679a16e60436 [term 1 LEADER]: Becoming Leader. State: Replica: 2e9e6fe7c03c4f2680c9679a16e60436, State: Running, Role: LEADER
I20260923 15:04:12.756163 26877 consensus_queue.cc:237] T 260cf94d3706497ea8436fac38ac3c45 P 2e9e6fe7c03c4f2680c9679a16e60436 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2301dc7034fb46b8bb6165fd0e620ba2" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 39955 } } peers { permanent_uuid: "5d5b6b7baf20426488ded3114f167e34" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 44109 } } peers { permanent_uuid: "2e9e6fe7c03c4f2680c9679a16e60436" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 35677 } }
I20260923 15:04:12.756956 26370 catalog_manager.cc:5495] T 260cf94d3706497ea8436fac38ac3c45 P 2e9e6fe7c03c4f2680c9679a16e60436 reported cstate change: term changed from 0 to 1, leader changed from <none> to 2e9e6fe7c03c4f2680c9679a16e60436 (127.13.75.129). New cstate: current_term: 1 leader_uuid: "2e9e6fe7c03c4f2680c9679a16e60436" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "2301dc7034fb46b8bb6165fd0e620ba2" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 39955 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5d5b6b7baf20426488ded3114f167e34" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 44109 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2e9e6fe7c03c4f2680c9679a16e60436" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 35677 } health_report { overall_health: HEALTHY } } }
I20260923 15:04:12.760540 26880 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:12.748695 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.131:39955 (local address 127.13.75.129:55979)
0923 15:04:12.749008 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0923 15:04:12.749020 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 15:04:12.749073 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:04:12.753952 (+ 4879us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:04:12.753953 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:04:12.753957 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:04:12.754022 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:04:12.754024 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:04:12.755071 (+ 1047us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:04:12.755072 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:04:12.759622 (+ 4550us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:04:12.759626 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:04:12.760241 (+ 615us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:04:12.760242 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:04:12.760319 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:04:12.760323 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 15:04:12.760400 (+ 77us) client_negotiation.cc:240] Negotiation successful
0923 15:04:12.760450 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":291,"thread_start_us":105,"threads_started":1}
I20260923 15:04:12.760556 26865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:12.748915 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.129:55979 (local address 127.13.75.131:39955)
0923 15:04:12.749085 (+ 170us) server_negotiation.cc:206] Beginning negotiation
0923 15:04:12.749087 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:04:12.749100 (+ 13us) server_negotiation.cc:407] Connection header received
0923 15:04:12.749134 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:04:12.749135 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:04:12.749148 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:04:12.749216 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:04:12.754135 (+ 4919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:04:12.754959 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:04:12.759719 (+ 4760us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:04:12.760139 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:04:12.760241 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:04:12.760251 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0923 15:04:12.760446 (+ 195us) server_negotiation.cc:299] Negotiation successful
0923 15:04:12.760494 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137}
I20260923 15:04:12.760991 26790 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "260cf94d3706497ea8436fac38ac3c45" candidate_uuid: "2e9e6fe7c03c4f2680c9679a16e60436" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2301dc7034fb46b8bb6165fd0e620ba2" is_pre_election: true
I20260923 15:04:12.761014 26789 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "260cf94d3706497ea8436fac38ac3c45" candidate_uuid: "2e9e6fe7c03c4f2680c9679a16e60436" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2301dc7034fb46b8bb6165fd0e620ba2"
I20260923 15:04:12.761125 26789 raft_consensus.cc:3037] T 260cf94d3706497ea8436fac38ac3c45 P 2301dc7034fb46b8bb6165fd0e620ba2 [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:04:12.761940 26789 raft_consensus.cc:2454] T 260cf94d3706497ea8436fac38ac3c45 P 2301dc7034fb46b8bb6165fd0e620ba2 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2e9e6fe7c03c4f2680c9679a16e60436 in term 1.
W20260923 15:04:12.779562 26703 tablet.cc:2369] T 260cf94d3706497ea8436fac38ac3c45 P 5d5b6b7baf20426488ded3114f167e34: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 15:04:12.814862 26866 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:12.811085 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45322 (local address 127.13.75.129:35677)
0923 15:04:12.811137 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0923 15:04:12.811139 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:04:12.811150 (+ 11us) server_negotiation.cc:407] Connection header received
0923 15:04:12.811209 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:04:12.811210 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:04:12.811237 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:04:12.811343 (+ 106us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 15:04:12.811573 (+ 230us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:04:12.812384 (+ 811us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:04:12.813309 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:04:12.813619 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:04:12.813701 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:04:12.813947 (+ 246us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 15:04:12.814100 (+ 153us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 15:04:12.814214 (+ 114us) server_negotiation.cc:1035] Waiting for connection context
0923 15:04:12.814743 (+ 529us) server_negotiation.cc:299] Negotiation successful
0923 15:04:12.814773 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260923 15:04:12.817220 26655 raft_consensus.cc:1261] T 260cf94d3706497ea8436fac38ac3c45 P 5d5b6b7baf20426488ded3114f167e34 [term 1 FOLLOWER]: Refusing update from remote peer 2e9e6fe7c03c4f2680c9679a16e60436: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 15:04:12.817622 26877 consensus_queue.cc:1035] T 260cf94d3706497ea8436fac38ac3c45 P 2e9e6fe7c03c4f2680c9679a16e60436 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5d5b6b7baf20426488ded3114f167e34" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 44109 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 15:04:12.820470 26789 raft_consensus.cc:1261] T 260cf94d3706497ea8436fac38ac3c45 P 2301dc7034fb46b8bb6165fd0e620ba2 [term 1 FOLLOWER]: Refusing update from remote peer 2e9e6fe7c03c4f2680c9679a16e60436: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 15:04:12.820763 26877 consensus_queue.cc:1035] T 260cf94d3706497ea8436fac38ac3c45 P 2e9e6fe7c03c4f2680c9679a16e60436 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2301dc7034fb46b8bb6165fd0e620ba2" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 39955 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 15:04:12.825599 26885 mvcc.cc:205] Tried to move back new op lower bound from 7332560293135941632 to 7332560292889780224. Current Snapshot: MvccSnapshot[applied={T|T < 7332560293135941632}]
I20260923 15:04:12.825836 26888 mvcc.cc:205] Tried to move back new op lower bound from 7332560293135941632 to 7332560292889780224. Current Snapshot: MvccSnapshot[applied={T|T < 7332560293135941632}]
I20260923 15:04:12.831466 26886 mvcc.cc:205] Tried to move back new op lower bound from 7332560293135941632 to 7332560292889780224. Current Snapshot: MvccSnapshot[applied={T|T < 7332560293135941632}]
I20260923 15:04:12.833595 26370 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38616:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 15:04:12.833719 26370 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:38616:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 15:04:12.834491 26370 catalog_manager.cc:5782] T 00000000000000000000000000000000 P dbc551ca623f4882ab0ac309072be9f0: Sending DeleteTablet for 3 replicas of tablet 260cf94d3706497ea8436fac38ac3c45
I20260923 15:04:12.834911 26770 tablet_service.cc:1514] Processing DeleteTablet for tablet 260cf94d3706497ea8436fac38ac3c45 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 15:04:12 UTC) from {username='kudu', principal='kudu/127.13.75.190@KRBTEST.COM'} at 127.0.0.1:38956
I20260923 15:04:12.834966 26484 tablet_service.cc:1514] Processing DeleteTablet for tablet 260cf94d3706497ea8436fac38ac3c45 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 15:04:12 UTC) from {username='kudu', principal='kudu/127.13.75.190@KRBTEST.COM'} at 127.0.0.1:45320
I20260923 15:04:12.835117 26636 tablet_service.cc:1514] Processing DeleteTablet for tablet 260cf94d3706497ea8436fac38ac3c45 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 15:04:12 UTC) from {username='kudu', principal='kudu/127.13.75.190@KRBTEST.COM'} at 127.0.0.1:45478
I20260923 15:04:12.835191 26894 tablet_replica.cc:330] T 260cf94d3706497ea8436fac38ac3c45 P 2301dc7034fb46b8bb6165fd0e620ba2: stopping tablet replica
I20260923 15:04:12.835304 26894 raft_consensus.cc:2229] T 260cf94d3706497ea8436fac38ac3c45 P 2301dc7034fb46b8bb6165fd0e620ba2 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 15:04:12.835427 26894 raft_consensus.cc:2258] T 260cf94d3706497ea8436fac38ac3c45 P 2301dc7034fb46b8bb6165fd0e620ba2 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 15:04:12.835690 13614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKC4soy/build/release/bin/kudu with pid 26408
I20260923 15:04:12.836907 26894 ts_tablet_manager.cc:1905] T 260cf94d3706497ea8436fac38ac3c45 P 2301dc7034fb46b8bb6165fd0e620ba2: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 15:04:12.839671 26894 ts_tablet_manager.cc:1918] T 260cf94d3706497ea8436fac38ac3c45 P 2301dc7034fb46b8bb6165fd0e620ba2: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 15:04:12.839774 26894 log.cc:1196] T 260cf94d3706497ea8436fac38ac3c45 P 2301dc7034fb46b8bb6165fd0e620ba2: Deleting WAL directory at /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/minicluster-data/ts-2/wal/wals/260cf94d3706497ea8436fac38ac3c45
I20260923 15:04:12.840121 26894 ts_tablet_manager.cc:1939] T 260cf94d3706497ea8436fac38ac3c45 P 2301dc7034fb46b8bb6165fd0e620ba2: Deleting consensus metadata
I20260923 15:04:12.840978 26357 catalog_manager.cc:4841] TS 2301dc7034fb46b8bb6165fd0e620ba2 (127.13.75.131:39955): tablet 260cf94d3706497ea8436fac38ac3c45 (table test-table [id=be430e9abbfa468493753f1d1ec40769]) successfully deleted
I20260923 15:04:12.847043 26896 tablet_replica.cc:330] T 260cf94d3706497ea8436fac38ac3c45 P 5d5b6b7baf20426488ded3114f167e34: stopping tablet replica
I20260923 15:04:12.847244 26896 raft_consensus.cc:2229] T 260cf94d3706497ea8436fac38ac3c45 P 5d5b6b7baf20426488ded3114f167e34 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 15:04:12.847406 26896 raft_consensus.cc:2258] T 260cf94d3706497ea8436fac38ac3c45 P 5d5b6b7baf20426488ded3114f167e34 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 15:04:12.848055 26896 ts_tablet_manager.cc:1905] T 260cf94d3706497ea8436fac38ac3c45 P 5d5b6b7baf20426488ded3114f167e34: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 15:04:12.850234 26896 ts_tablet_manager.cc:1918] T 260cf94d3706497ea8436fac38ac3c45 P 5d5b6b7baf20426488ded3114f167e34: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 15:04:12.850349 26896 log.cc:1196] T 260cf94d3706497ea8436fac38ac3c45 P 5d5b6b7baf20426488ded3114f167e34: Deleting WAL directory at /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers.1790175691491353-13614-0/minicluster-data/ts-1/wal/wals/260cf94d3706497ea8436fac38ac3c45
I20260923 15:04:12.850620 26896 ts_tablet_manager.cc:1939] T 260cf94d3706497ea8436fac38ac3c45 P 5d5b6b7baf20426488ded3114f167e34: Deleting consensus metadata
I20260923 15:04:12.851509 26355 catalog_manager.cc:4841] TS 5d5b6b7baf20426488ded3114f167e34 (127.13.75.130:44109): tablet 260cf94d3706497ea8436fac38ac3c45 (table test-table [id=be430e9abbfa468493753f1d1ec40769]) successfully deleted
W20260923 15:04:12.855275 26354 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.75.129:35677 (error 108)
W20260923 15:04:12.855990 26867 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 15:04:12.855786 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.129:35677 (local address 127.0.0.1:45328)
0923 15:04:12.855851 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0923 15:04:12.855864 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0923 15:04:12.855934 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.129:35677: BlockingWrite error: write error: Connection reset by peer (error 104)
Metrics: {"client-negotiator.queue_time_us":42}
W20260923 15:04:12.856106 26354 catalog_manager.cc:4584] TS 2e9e6fe7c03c4f2680c9679a16e60436 (127.13.75.129:35677): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 260cf94d3706497ea8436fac38ac3c45: Network error: Client connection negotiation failed: client connection to 127.13.75.129:35677: BlockingWrite error: write error: Connection reset by peer (error 104)
I20260923 15:04:12.856439 13614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKC4soy/build/release/bin/kudu with pid 26572
I20260923 15:04:12.864854 13614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKC4soy/build/release/bin/kudu with pid 26704
I20260923 15:04:12.872870 13614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKC4soy/build/release/bin/kudu with pid 26337
I20260923 15:04:12.878842 13614 mini_ranger_kms.cc:60] Stopping Ranger KMS...
I20260923 15:04:13.223816 13614 mini_ranger_kms.cc:62] Stopped Ranger KMS
I20260923 15:04:13.223924 13614 mini_ranger.cc:66] Stopping Ranger...
I20260923 15:04:13.569051 13614 mini_ranger.cc:68] Stopped Ranger
2026-09-23 15:04:13.569 UTC [25403] LOG: received smart shutdown request
2026-09-23 15:04:13.574 UTC [25403] LOG: background worker "logical replication launcher" (PID 25412) exited with exit code 1
2026-09-23 15:04:13.574 UTC [25407] LOG: shutting down
2026-09-23 15:04:13.778 UTC [25403] LOG: database system is shut down
2026-09-23T15:04:13Z chronyd exiting
[ OK ] SecurityITest.TestEncryptionWithKMSIntegrationMultipleServers (51067 ms)
[ RUN ] SecurityITest.IPKICACert
Loading random data
Initializing database '/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790175691491353-13614-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 15:04:13 dist-test-slave-cnb0 krb5kdc[26907](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 15:04:13 dist-test-slave-cnb0 krb5kdc[26907](info): set up 2 sockets
Sep 23 15:04:13 dist-test-slave-cnb0 krb5kdc[26907](info): commencing operation
krb5kdc: starting...
W20260923 15:04:14.947899 13614 mini_kdc.cc:121] Time spent starting KDC: real 1.041s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790175691491353-13614-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790175691491353-13614-0/krb5kdc/test-user.keytab.
Sep 23 15:04:15 dist-test-slave-cnb0 krb5kdc[26907](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175855, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T15:04:15Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T15:04:15Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.75.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.75.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.75.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.190.keytab.
Entry for principal kudu/127.13.75.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.190.keytab.
I20260923 15:04:15.066996 13614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKC4soy/build/release/bin/kudu
/tmp/dist-test-taskKC4soy/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790175691491353-13614-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790175691491353-13614-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790175691491353-13614-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790175691491353-13614-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.75.190:40393
--webserver_interface=127.13.75.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.190.keytab
--principal=kudu/127.13.75.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.75.148:44231
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.75.190:40393,127.13.75.189:39335,127.13.75.188:45931
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790175691491353-13614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790175691491353-13614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790175691491353-13614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:04:15.153162 26920 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:04:15.153388 26920 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:04:15.153412 26920 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:04:15.154583 26920 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 15:04:15.154627 26920 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:04:15.154642 26920 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:04:15.154654 26920 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 15:04:15.154667 26920 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 15:04:15.156275 26920 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.75.148:44231
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790175691491353-13614-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790175691491353-13614-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.75.190:40393,127.13.75.189:39335,127.13.75.188:45931
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.190.keytab
--principal=kudu/127.13.75.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.75.190:40393
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790175691491353-13614-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.75.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26920
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790175691491353-13614-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:04:15.156554 26920 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:04:15.156800 26920 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:04:15.161927 26925 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:04:15.161880 26928 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:04:15.162038 26920 server_base.cc:1034] running on GCE node
W20260923 15:04:15.161880 26926 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:04:15.162374 26920 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:04:15.162688 26920 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:04:15.163874 26920 hybrid_clock.cc:648] HybridClock initialized: now 1790175855163835 us; error 54 us; skew 500 ppm
Sep 23 15:04:15 dist-test-slave-cnb0 krb5kdc[26907](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175855, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:04:15.166613 26920 init.cc:377] Logged in from keytab as kudu/127.13.75.190@KRBTEST.COM (short username kudu)
I20260923 15:04:15.167141 26920 webserver.cc:460] Webserver started at http://127.13.75.190:46407/ using document root <none> and password file <none>
I20260923 15:04:15.167387 26920 fs_manager.cc:359] Metadata directory not provided
I20260923 15:04:15.167460 26920 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:04:15.167608 26920 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:04:15.168671 26920 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790175691491353-13614-0/minicluster-data/master-0/data/instance:
uuid: "d3dfecc8fcf041ce8e9fcda3eb907ada"
format_stamp: "Formatted at 2026-09-23 15:04:15 on dist-test-slave-cnb0"
server_key: "f431cb9ce2ce630bf1f408f1b8ed44a3"
server_key_iv: "4174764d1b61f58d6992434f02c49f9e"
server_key_version: "encryptionkey@0"
I20260923 15:04:15.169035 26920 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790175691491353-13614-0/minicluster-data/master-0/wal/instance:
uuid: "d3dfecc8fcf041ce8e9fcda3eb907ada"
format_stamp: "Formatted at 2026-09-23 15:04:15 on dist-test-slave-cnb0"
server_key: "f431cb9ce2ce630bf1f408f1b8ed44a3"
server_key_iv: "4174764d1b61f58d6992434f02c49f9e"
server_key_version: "encryptionkey@0"
I20260923 15:04:15.170332 26920 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 15:04:15.171164 26935 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:04:15.171362 26920 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:04:15.171480 26920 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790175691491353-13614-0/minicluster-data/master-0/data,/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790175691491353-13614-0/minicluster-data/master-0/wal
uuid: "d3dfecc8fcf041ce8e9fcda3eb907ada"
format_stamp: "Formatted at 2026-09-23 15:04:15 on dist-test-slave-cnb0"
server_key: "f431cb9ce2ce630bf1f408f1b8ed44a3"
server_key_iv: "4174764d1b61f58d6992434f02c49f9e"
server_key_version: "encryptionkey@0"
I20260923 15:04:15.171562 26920 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790175691491353-13614-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790175691491353-13614-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790175691491353-13614-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:04:15.208323 26920 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:04:15.210722 26920 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:04:15.210944 26920 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:04:15.214951 26987 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.75.190:40393 every 8 connection(s)
I20260923 15:04:15.214959 26920 rpc_server.cc:307] RPC server started. Bound to: 127.13.75.190:40393
I20260923 15:04:15.215390 26920 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790175691491353-13614-0/minicluster-data/master-0/data/info.pb
I20260923 15:04:15.216259 26988 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:04:15.217846 26988 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.75.190" port: 40393 } has no permanent_uuid. Determining permanent_uuid...
I20260923 15:04:15.222417 13614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKC4soy/build/release/bin/kudu as pid 26920
I20260923 15:04:15.222555 13614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790175691491353-13614-0/minicluster-data/master-0/wal/instance
I20260923 15:04:15.222752 13614 external_mini_cluster.cc:1428] Setting key de1be1b6c8e44921dbde22db92c76e89
Sep 23 15:04:15 dist-test-slave-cnb0 krb5kdc[26907](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175855, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.190@KRBTEST.COM for kudu/127.13.75.190@KRBTEST.COM
I20260923 15:04:15.228403 26990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:15.218521 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:40393 (local address 127.0.0.1:49978)
0923 15:04:15.218684 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0923 15:04:15.218719 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0923 15:04:15.219238 (+ 519us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:04:15.219573 (+ 335us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:04:15.219575 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:04:15.219838 (+ 263us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:04:15.219996 (+ 158us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:04:15.220000 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:04:15.221003 (+ 1003us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:04:15.221004 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:04:15.221705 (+ 701us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:04:15.221708 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:04:15.222193 (+ 485us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:04:15.222194 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:04:15.222250 (+ 56us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:04:15.222862 (+ 612us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:04:15.222872 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:04:15.224550 (+ 1678us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:04:15.226781 (+ 2231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:04:15.226791 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:04:15.226793 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:04:15.227106 (+ 313us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:04:15.227385 (+ 279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:04:15.227385 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:04:15.227386 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:04:15.227477 (+ 91us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:04:15.227977 (+ 500us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:04:15.227982 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:04:15.228040 (+ 58us) client_negotiation.cc:769] Sending connection context
0923 15:04:15.228108 (+ 68us) client_negotiation.cc:240] Negotiation successful
0923 15:04:15.228230 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":80,"threads_started":1}
I20260923 15:04:15.228492 26991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:15.218689 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49978 (local address 127.13.75.190:40393)
0923 15:04:15.218837 (+ 148us) server_negotiation.cc:206] Beginning negotiation
0923 15:04:15.218841 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 15:04:15.218927 (+ 86us) server_negotiation.cc:407] Connection header received
0923 15:04:15.219437 (+ 510us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:04:15.219440 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:04:15.219461 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:04:15.219545 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:04:15.220081 (+ 536us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:04:15.220914 (+ 833us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:04:15.221782 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:04:15.222113 (+ 331us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:04:15.222249 (+ 136us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:04:15.224663 (+ 2414us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:04:15.224670 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:04:15.224670 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:04:15.224678 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:04:15.226679 (+ 2001us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:04:15.227196 (+ 517us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:04:15.227200 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:04:15.227201 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:04:15.227310 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:04:15.227556 (+ 246us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:04:15.227557 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:04:15.227558 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:04:15.227896 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:04:15.227992 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0923 15:04:15.228193 (+ 201us) server_negotiation.cc:299] Negotiation successful
0923 15:04:15.228315 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":5,"server-negotiator.queue_time_us":117,"thread_start_us":89,"threads_started":1}
I20260923 15:04:15.229277 26988 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.75.189" port: 39335 } has no permanent_uuid. Determining permanent_uuid...
I20260923 15:04:15.229642 26990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:15.229522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.189:39335 (local address 127.0.0.1:37078)
0923 15:04:15.229575 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0923 15:04:15.229623 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.189:39335: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260923 15:04:15.229768 26936 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.13.75.189:39335: connect: Connection refused (error 111)
I20260923 15:04:15.230213 26990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:15.230115 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.189:39335 (local address 127.0.0.1:37094)
0923 15:04:15.230160 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0923 15:04:15.230185 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.189:39335: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260923 15:04:15.230350 26988 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.75.189:39335: Network error: Client connection negotiation failed: client connection to 127.13.75.189:39335: connect: Connection refused (error 111)
WARNING: no policy specified for kudu/127.13.75.189@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.75.189@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.75.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.189.keytab.
Entry for principal kudu/127.13.75.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.189.keytab.
I20260923 15:04:15.250500 13614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKC4soy/build/release/bin/kudu
/tmp/dist-test-taskKC4soy/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790175691491353-13614-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790175691491353-13614-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790175691491353-13614-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790175691491353-13614-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.75.189:39335
--webserver_interface=127.13.75.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.189.keytab
--principal=kudu/127.13.75.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.75.148:44231
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.75.190:40393,127.13.75.189:39335,127.13.75.188:45931
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790175691491353-13614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790175691491353-13614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790175691491353-13614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260923 15:04:15.279574 26988 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.75.189" port: 39335 } attempt: 1
I20260923 15:04:15.280158 26990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:15.279997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.189:39335 (local address 127.0.0.1:37100)
0923 15:04:15.280069 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0923 15:04:15.280121 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.189:39335: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260923 15:04:15.280555 26990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:15.280442 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.189:39335 (local address 127.0.0.1:37106)
0923 15:04:15.280497 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0923 15:04:15.280531 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.189:39335: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
W20260923 15:04:15.280732 26988 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.75.189:39335: Network error: Client connection negotiation failed: client connection to 127.13.75.189:39335: connect: Connection refused (error 111)
W20260923 15:04:15.334329 26995 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:04:15.334633 26995 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:04:15.334661 26995 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:04:15.335862 26995 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 15:04:15.335909 26995 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:04:15.335924 26995 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:04:15.335937 26995 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 15:04:15.335949 26995 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 15:04:15.337507 26995 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.75.148:44231
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790175691491353-13614-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790175691491353-13614-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.13.75.190:40393,127.13.75.189:39335,127.13.75.188:45931
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.189.keytab
--principal=kudu/127.13.75.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.75.189:39335
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790175691491353-13614-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.13.75.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.26995
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790175691491353-13614-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:04:15.337771 26995 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:04:15.338020 26995 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:04:15.343037 27000 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:04:15.343041 27003 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:04:15.343040 27001 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:04:15.343062 26995 server_base.cc:1034] running on GCE node
I20260923 15:04:15.343518 26995 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:04:15.343772 26995 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:04:15.344945 26995 hybrid_clock.cc:648] HybridClock initialized: now 1790175855344923 us; error 34 us; skew 500 ppm
Sep 23 15:04:15 dist-test-slave-cnb0 krb5kdc[26907](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175855, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:04:15.347630 26995 init.cc:377] Logged in from keytab as kudu/127.13.75.189@KRBTEST.COM (short username kudu)
I20260923 15:04:15.348081 26995 webserver.cc:460] Webserver started at http://127.13.75.189:34221/ using document root <none> and password file <none>
I20260923 15:04:15.348340 26995 fs_manager.cc:359] Metadata directory not provided
I20260923 15:04:15.348465 26995 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:04:15.348618 26995 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:04:15.348943 26988 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.75.189" port: 39335 } attempt: 2
I20260923 15:04:15.349341 26990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:15.349211 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.189:39335 (local address 127.0.0.1:37122)
0923 15:04:15.349273 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0923 15:04:15.349310 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.189:39335: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260923 15:04:15.349722 26990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:15.349635 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.189:39335 (local address 127.0.0.1:37126)
0923 15:04:15.349681 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0923 15:04:15.349706 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.189:39335: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260923 15:04:15.349833 26988 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.75.189:39335: Network error: Client connection negotiation failed: client connection to 127.13.75.189:39335: connect: Connection refused (error 111)
I20260923 15:04:15.350040 26995 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790175691491353-13614-0/minicluster-data/master-1/data/instance:
uuid: "ee6df7ad5f004ac0a9680e393c0d9ff5"
format_stamp: "Formatted at 2026-09-23 15:04:15 on dist-test-slave-cnb0"
server_key: "2773ba939ff8323bfa9723a9e1eb33d6"
server_key_iv: "33633517baef5f50dae21da4c2ac44d4"
server_key_version: "encryptionkey@0"
I20260923 15:04:15.350442 26995 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790175691491353-13614-0/minicluster-data/master-1/wal/instance:
uuid: "ee6df7ad5f004ac0a9680e393c0d9ff5"
format_stamp: "Formatted at 2026-09-23 15:04:15 on dist-test-slave-cnb0"
server_key: "2773ba939ff8323bfa9723a9e1eb33d6"
server_key_iv: "33633517baef5f50dae21da4c2ac44d4"
server_key_version: "encryptionkey@0"
I20260923 15:04:15.351890 26995 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.004s sys 0.000s
I20260923 15:04:15.352684 27010 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:04:15.352876 26995 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 15:04:15.352952 26995 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790175691491353-13614-0/minicluster-data/master-1/data,/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790175691491353-13614-0/minicluster-data/master-1/wal
uuid: "ee6df7ad5f004ac0a9680e393c0d9ff5"
format_stamp: "Formatted at 2026-09-23 15:04:15 on dist-test-slave-cnb0"
server_key: "2773ba939ff8323bfa9723a9e1eb33d6"
server_key_iv: "33633517baef5f50dae21da4c2ac44d4"
server_key_version: "encryptionkey@0"
I20260923 15:04:15.353008 26995 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790175691491353-13614-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790175691491353-13614-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790175691491353-13614-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:04:15.359843 26995 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:04:15.361802 26995 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:04:15.361913 26995 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:04:15.365867 26995 rpc_server.cc:307] RPC server started. Bound to: 127.13.75.189:39335
I20260923 15:04:15.365904 27062 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.75.189:39335 every 8 connection(s)
I20260923 15:04:15.366302 26995 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790175691491353-13614-0/minicluster-data/master-1/data/info.pb
I20260923 15:04:15.367189 27063 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:04:15.368757 27063 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.75.190" port: 40393 } has no permanent_uuid. Determining permanent_uuid...
Sep 23 15:04:15 dist-test-slave-cnb0 krb5kdc[26907](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175855, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.189@KRBTEST.COM for kudu/127.13.75.190@KRBTEST.COM
I20260923 15:04:15.376245 13614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKC4soy/build/release/bin/kudu as pid 26995
I20260923 15:04:15.376374 13614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790175691491353-13614-0/minicluster-data/master-1/wal/instance
I20260923 15:04:15.376588 13614 external_mini_cluster.cc:1428] Setting key 0d5990b9b5d21811d0bd0983cbc119fc
I20260923 15:04:15.378345 26991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:15.369467 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49986 (local address 127.13.75.190:40393)
0923 15:04:15.369541 (+ 74us) server_negotiation.cc:206] Beginning negotiation
0923 15:04:15.369543 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:04:15.369725 (+ 182us) server_negotiation.cc:407] Connection header received
0923 15:04:15.370153 (+ 428us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:04:15.370153 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:04:15.370165 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:04:15.370232 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:04:15.370802 (+ 570us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:04:15.371406 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:04:15.372432 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:04:15.372651 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:04:15.372712 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:04:15.375099 (+ 2387us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:04:15.375101 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:04:15.375101 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:04:15.375109 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:04:15.376860 (+ 1751us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:04:15.377266 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:04:15.377267 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:04:15.377268 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:04:15.377352 (+ 84us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:04:15.377601 (+ 249us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:04:15.377601 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:04:15.377602 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:04:15.377777 (+ 175us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:04:15.377852 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0923 15:04:15.378153 (+ 301us) server_negotiation.cc:299] Negotiation successful
0923 15:04:15.378261 (+ 108us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260923 15:04:15.378577 27065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:15.369429 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:40393 (local address 127.0.0.1:49986)
0923 15:04:15.369595 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0923 15:04:15.369638 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0923 15:04:15.370074 (+ 436us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:04:15.370284 (+ 210us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:04:15.370287 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:04:15.370541 (+ 254us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:04:15.370729 (+ 188us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:04:15.370734 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:04:15.371517 (+ 783us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:04:15.371518 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:04:15.372323 (+ 805us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:04:15.372326 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:04:15.372750 (+ 424us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:04:15.372751 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:04:15.372863 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:04:15.373418 (+ 555us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:04:15.373426 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:04:15.374978 (+ 1552us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:04:15.376953 (+ 1975us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:04:15.376959 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:04:15.376962 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:04:15.377187 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:04:15.377430 (+ 243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:04:15.377430 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:04:15.377431 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:04:15.377534 (+ 103us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:04:15.377845 (+ 311us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:04:15.377855 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:04:15.377977 (+ 122us) client_negotiation.cc:769] Sending connection context
0923 15:04:15.378107 (+ 130us) client_negotiation.cc:240] Negotiation successful
0923 15:04:15.378304 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":82,"threads_started":1}
I20260923 15:04:15.379355 27063 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.75.189" port: 39335 } has no permanent_uuid. Determining permanent_uuid...
Sep 23 15:04:15 dist-test-slave-cnb0 krb5kdc[26907](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175855, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.189@KRBTEST.COM for kudu/127.13.75.189@KRBTEST.COM
WARNING: no policy specified for kudu/127.13.75.188@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.75.188@KRBTEST.COM" created.
I20260923 15:04:15.387868 27065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:15.379623 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.189:39335 (local address 127.0.0.1:37142)
0923 15:04:15.379682 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0923 15:04:15.379694 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 15:04:15.379740 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:04:15.380008 (+ 268us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:04:15.380009 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:04:15.380170 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:04:15.380234 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:04:15.380236 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:04:15.381245 (+ 1009us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:04:15.381246 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:04:15.382165 (+ 919us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:04:15.382168 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:04:15.382716 (+ 548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:04:15.382717 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:04:15.382803 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:04:15.383250 (+ 447us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:04:15.383256 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:04:15.384591 (+ 1335us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:04:15.386511 (+ 1920us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:04:15.386512 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:04:15.386513 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:04:15.386674 (+ 161us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:04:15.386956 (+ 282us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:04:15.386957 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:04:15.386957 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:04:15.387028 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:04:15.387318 (+ 290us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:04:15.387319 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:04:15.387408 (+ 89us) client_negotiation.cc:769] Sending connection context
0923 15:04:15.387508 (+ 100us) client_negotiation.cc:240] Negotiation successful
0923 15:04:15.387638 (+ 130us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260923 15:04:15.387856 27067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:15.379725 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37142 (local address 127.13.75.189:39335)
0923 15:04:15.379861 (+ 136us) server_negotiation.cc:206] Beginning negotiation
0923 15:04:15.379878 (+ 17us) server_negotiation.cc:399] Waiting for connection header
0923 15:04:15.379894 (+ 16us) server_negotiation.cc:407] Connection header received
0923 15:04:15.379912 (+ 18us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:04:15.379913 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:04:15.379924 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:04:15.379981 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:04:15.380311 (+ 330us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:04:15.381151 (+ 840us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:04:15.382300 (+ 1149us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:04:15.382630 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:04:15.382693 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:04:15.384700 (+ 2007us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:04:15.384709 (+ 9us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:04:15.384709 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:04:15.384716 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:04:15.386406 (+ 1690us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:04:15.386759 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:04:15.386762 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:04:15.386763 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:04:15.386823 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:04:15.387098 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:04:15.387098 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:04:15.387099 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:04:15.387249 (+ 150us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:04:15.387324 (+ 75us) server_negotiation.cc:1035] Waiting for connection context
0923 15:04:15.387560 (+ 236us) server_negotiation.cc:299] Negotiation successful
0923 15:04:15.387712 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":51,"server-negotiator.queue_time_us":112,"thread_start_us":84,"threads_started":1}
I20260923 15:04:15.388689 27063 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.75.188" port: 45931 } has no permanent_uuid. Determining permanent_uuid...
I20260923 15:04:15.389075 27065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:15.388942 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.188:45931 (local address 127.0.0.1:39946)
0923 15:04:15.388991 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0923 15:04:15.389043 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.188:45931: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260923 15:04:15.389206 27011 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.13.75.188:45931: connect: Connection refused (error 111)
I20260923 15:04:15.389604 27065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:15.389510 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.188:45931 (local address 127.0.0.1:39950)
0923 15:04:15.389559 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0923 15:04:15.389583 (+ 24us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.188:45931: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260923 15:04:15.389716 27063 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.75.188:45931: Network error: Client connection negotiation failed: client connection to 127.13.75.188:45931: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.75.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.188.keytab.
Entry for principal kudu/127.13.75.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.188.keytab.
I20260923 15:04:15.402410 13614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKC4soy/build/release/bin/kudu
/tmp/dist-test-taskKC4soy/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790175691491353-13614-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790175691491353-13614-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790175691491353-13614-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790175691491353-13614-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.75.188:45931
--webserver_interface=127.13.75.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.188.keytab
--principal=kudu/127.13.75.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.75.148:44231
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.75.190:40393,127.13.75.189:39335,127.13.75.188:45931
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790175691491353-13614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790175691491353-13614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790175691491353-13614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260923 15:04:15.438941 27063 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.75.188" port: 45931 } attempt: 1
I20260923 15:04:15.439481 27065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:15.439309 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.188:45931 (local address 127.0.0.1:39964)
0923 15:04:15.439386 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0923 15:04:15.439439 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.188:45931: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":44}
I20260923 15:04:15.439881 27065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:15.439777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.188:45931 (local address 127.0.0.1:39972)
0923 15:04:15.439825 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0923 15:04:15.439858 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.188:45931: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260923 15:04:15.440017 27063 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.75.188:45931: Network error: Client connection negotiation failed: client connection to 127.13.75.188:45931: connect: Connection refused (error 111)
I20260923 15:04:15.441009 26988 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.75.189" port: 39335 } attempt: 3
Sep 23 15:04:15 dist-test-slave-cnb0 krb5kdc[26907](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175855, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.190@KRBTEST.COM for kudu/127.13.75.189@KRBTEST.COM
I20260923 15:04:15.448026 26990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:15.441271 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.189:39335 (local address 127.0.0.1:37154)
0923 15:04:15.441326 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0923 15:04:15.441340 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0923 15:04:15.441410 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:04:15.441570 (+ 160us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:04:15.441571 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:04:15.441732 (+ 161us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:04:15.441800 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:04:15.441802 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:04:15.442550 (+ 748us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:04:15.442551 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:04:15.443130 (+ 579us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:04:15.443133 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:04:15.443540 (+ 407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:04:15.443541 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:04:15.443623 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:04:15.444094 (+ 471us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:04:15.444099 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:04:15.445246 (+ 1147us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:04:15.446942 (+ 1696us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:04:15.446943 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:04:15.446943 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:04:15.447129 (+ 186us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:04:15.447336 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:04:15.447337 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:04:15.447337 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:04:15.447398 (+ 61us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:04:15.447679 (+ 281us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:04:15.447680 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:04:15.447728 (+ 48us) client_negotiation.cc:769] Sending connection context
0923 15:04:15.447818 (+ 90us) client_negotiation.cc:240] Negotiation successful
0923 15:04:15.447929 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260923 15:04:15.448026 27067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:15.441341 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37154 (local address 127.13.75.189:39335)
0923 15:04:15.441404 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0923 15:04:15.441406 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:04:15.441418 (+ 12us) server_negotiation.cc:407] Connection header received
0923 15:04:15.441478 (+ 60us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:04:15.441479 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:04:15.441488 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:04:15.441560 (+ 72us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:04:15.441891 (+ 331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:04:15.442446 (+ 555us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:04:15.443213 (+ 767us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:04:15.443448 (+ 235us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:04:15.443539 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:04:15.445335 (+ 1796us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:04:15.445347 (+ 12us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:04:15.445347 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:04:15.445359 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:04:15.446824 (+ 1465us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:04:15.447206 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:04:15.447207 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:04:15.447208 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:04:15.447266 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:04:15.447474 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:04:15.447475 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:04:15.447475 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:04:15.447602 (+ 127us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:04:15.447678 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0923 15:04:15.447844 (+ 166us) server_negotiation.cc:299] Negotiation successful
0923 15:04:15.447925 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260923 15:04:15.448431 26988 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.75.188" port: 45931 } has no permanent_uuid. Determining permanent_uuid...
I20260923 15:04:15.448819 26990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:15.448717 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.188:45931 (local address 127.0.0.1:39980)
0923 15:04:15.448771 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0923 15:04:15.448800 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.188:45931: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
I20260923 15:04:15.449117 26990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:15.449033 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.188:45931 (local address 127.0.0.1:39990)
0923 15:04:15.449076 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0923 15:04:15.449098 (+ 22us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.188:45931: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260923 15:04:15.449216 26988 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.75.188:45931: Network error: Client connection negotiation failed: client connection to 127.13.75.188:45931: connect: Connection refused (error 111)
I20260923 15:04:15.480366 26988 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.75.188" port: 45931 } attempt: 1
I20260923 15:04:15.480815 26990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:15.480684 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.188:45931 (local address 127.0.0.1:39996)
0923 15:04:15.480752 (+ 68us) negotiation.cc:107] Waiting for socket to connect
0923 15:04:15.480787 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.188:45931: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
I20260923 15:04:15.481150 26990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:15.481052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.188:45931 (local address 127.0.0.1:39998)
0923 15:04:15.481100 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0923 15:04:15.481127 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.188:45931: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260923 15:04:15.481232 26988 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.75.188:45931: Network error: Client connection negotiation failed: client connection to 127.13.75.188:45931: connect: Connection refused (error 111)
W20260923 15:04:15.488260 27070 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:04:15.488461 27070 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:04:15.488482 27070 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:04:15.489617 27070 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 15:04:15.489660 27070 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:04:15.489674 27070 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:04:15.489686 27070 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 15:04:15.489737 27070 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 15:04:15.491367 27070 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.75.148:44231
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790175691491353-13614-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790175691491353-13614-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.13.75.190:40393,127.13.75.189:39335,127.13.75.188:45931
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.188.keytab
--principal=kudu/127.13.75.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.75.188:45931
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790175691491353-13614-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.13.75.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27070
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790175691491353-13614-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:04:15.491608 27070 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:04:15.491834 27070 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:04:15.496626 27078 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:04:15.496626 27076 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:04:15.496845 27070 server_base.cc:1034] running on GCE node
W20260923 15:04:15.496636 27075 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:04:15.497159 27070 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:04:15.497463 27070 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:04:15.498620 27070 hybrid_clock.cc:648] HybridClock initialized: now 1790175855498607 us; error 33 us; skew 500 ppm
Sep 23 15:04:15 dist-test-slave-cnb0 krb5kdc[26907](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175855, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:04:15.501250 27070 init.cc:377] Logged in from keytab as kudu/127.13.75.188@KRBTEST.COM (short username kudu)
I20260923 15:04:15.501767 27070 webserver.cc:460] Webserver started at http://127.13.75.188:39945/ using document root <none> and password file <none>
I20260923 15:04:15.502022 27070 fs_manager.cc:359] Metadata directory not provided
I20260923 15:04:15.502091 27070 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:04:15.502240 27070 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:04:15.503398 27070 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790175691491353-13614-0/minicluster-data/master-2/data/instance:
uuid: "97d3c821ddd447cf996a6fd286e6060b"
format_stamp: "Formatted at 2026-09-23 15:04:15 on dist-test-slave-cnb0"
server_key: "5bc6172f70b5b309a0368c42738cfafc"
server_key_iv: "6120944a7a12a2c066f61a0597800294"
server_key_version: "encryptionkey@0"
I20260923 15:04:15.503762 27070 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790175691491353-13614-0/minicluster-data/master-2/wal/instance:
uuid: "97d3c821ddd447cf996a6fd286e6060b"
format_stamp: "Formatted at 2026-09-23 15:04:15 on dist-test-slave-cnb0"
server_key: "5bc6172f70b5b309a0368c42738cfafc"
server_key_iv: "6120944a7a12a2c066f61a0597800294"
server_key_version: "encryptionkey@0"
I20260923 15:04:15.505116 27070 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:04:15.505941 27085 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:04:15.506134 27070 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:04:15.506244 27070 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790175691491353-13614-0/minicluster-data/master-2/data,/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790175691491353-13614-0/minicluster-data/master-2/wal
uuid: "97d3c821ddd447cf996a6fd286e6060b"
format_stamp: "Formatted at 2026-09-23 15:04:15 on dist-test-slave-cnb0"
server_key: "5bc6172f70b5b309a0368c42738cfafc"
server_key_iv: "6120944a7a12a2c066f61a0597800294"
server_key_version: "encryptionkey@0"
I20260923 15:04:15.506320 27070 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790175691491353-13614-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790175691491353-13614-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790175691491353-13614-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:04:15.508173 27063 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.75.188" port: 45931 } attempt: 2
I20260923 15:04:15.508816 27065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:15.508663 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.188:45931 (local address 127.0.0.1:40004)
0923 15:04:15.508741 (+ 78us) negotiation.cc:107] Waiting for socket to connect
0923 15:04:15.508783 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.188:45931: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
I20260923 15:04:15.509158 27065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:15.509052 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.188:45931 (local address 127.0.0.1:40006)
0923 15:04:15.509101 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0923 15:04:15.509127 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.188:45931: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
W20260923 15:04:15.509245 27063 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.75.188:45931: Network error: Client connection negotiation failed: client connection to 127.13.75.188:45931: connect: Connection refused (error 111)
I20260923 15:04:15.517185 27070 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:04:15.519227 27070 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:04:15.519397 27070 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:04:15.523746 27070 rpc_server.cc:307] RPC server started. Bound to: 127.13.75.188:45931
I20260923 15:04:15.523801 27137 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.75.188:45931 every 8 connection(s)
I20260923 15:04:15.524158 27070 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790175691491353-13614-0/minicluster-data/master-2/data/info.pb
I20260923 15:04:15.525043 27138 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:04:15.526611 27138 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.75.190" port: 40393 } has no permanent_uuid. Determining permanent_uuid...
I20260923 15:04:15.527522 13614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKC4soy/build/release/bin/kudu as pid 27070
I20260923 15:04:15.527621 13614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.IPKICACert.1790175691491353-13614-0/minicluster-data/master-2/wal/instance
I20260923 15:04:15.527814 13614 external_mini_cluster.cc:1428] Setting key 71ec3d055a9f99238a1ca66859a6d0d6
Sep 23 15:04:15 dist-test-slave-cnb0 krb5kdc[26907](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175855, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.188@KRBTEST.COM for kudu/127.13.75.190@KRBTEST.COM
Sep 23 15:04:15 dist-test-slave-cnb0 krb5kdc[26907](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175855, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.75.188@KRBTEST.COM
I20260923 15:04:15.538722 26991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:15.527493 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49998 (local address 127.13.75.190:40393)
0923 15:04:15.527545 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0923 15:04:15.527547 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:04:15.527721 (+ 174us) server_negotiation.cc:407] Connection header received
0923 15:04:15.528461 (+ 740us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:04:15.528463 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:04:15.528476 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:04:15.528547 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:04:15.529342 (+ 795us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:04:15.530149 (+ 807us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:04:15.531678 (+ 1529us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:04:15.531986 (+ 308us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:04:15.532065 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:04:15.535268 (+ 3203us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:04:15.535270 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:04:15.535270 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:04:15.535277 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:04:15.536927 (+ 1650us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:04:15.537548 (+ 621us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:04:15.537550 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:04:15.537550 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:04:15.537610 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:04:15.537950 (+ 340us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:04:15.537951 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:04:15.537951 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:04:15.538089 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:04:15.538175 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0923 15:04:15.538523 (+ 348us) server_negotiation.cc:299] Negotiation successful
0923 15:04:15.538626 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260923 15:04:15.538970 27140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:15.527322 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:40393 (local address 127.0.0.1:49998)
0923 15:04:15.527563 (+ 241us) negotiation.cc:107] Waiting for socket to connect
0923 15:04:15.527618 (+ 55us) client_negotiation.cc:174] Beginning negotiation
0923 15:04:15.528299 (+ 681us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:04:15.528596 (+ 297us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:04:15.528599 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:04:15.528988 (+ 389us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:04:15.529220 (+ 232us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:04:15.529226 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:04:15.530285 (+ 1059us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:04:15.530287 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:04:15.531552 (+ 1265us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:04:15.531556 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:04:15.532149 (+ 593us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:04:15.532150 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:04:15.532264 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:04:15.533065 (+ 801us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:04:15.533075 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:04:15.535105 (+ 2030us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:04:15.537018 (+ 1913us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:04:15.537024 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:04:15.537026 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:04:15.537451 (+ 425us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:04:15.537706 (+ 255us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:04:15.537707 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:04:15.537707 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:04:15.537829 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:04:15.538165 (+ 336us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:04:15.538172 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:04:15.538329 (+ 157us) client_negotiation.cc:769] Sending connection context
0923 15:04:15.538484 (+ 155us) client_negotiation.cc:240] Negotiation successful
0923 15:04:15.538663 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":104,"threads_started":1}
I20260923 15:04:15.539065 27142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:15.528219 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40016 (local address 127.13.75.188:45931)
0923 15:04:15.528380 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0923 15:04:15.528384 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 15:04:15.528418 (+ 34us) server_negotiation.cc:407] Connection header received
0923 15:04:15.528498 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:04:15.528500 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:04:15.528521 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:04:15.528607 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:04:15.529356 (+ 749us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:04:15.530506 (+ 1150us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:04:15.531680 (+ 1174us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:04:15.532185 (+ 505us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:04:15.532261 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:04:15.535596 (+ 3335us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:04:15.535603 (+ 7us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:04:15.535604 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:04:15.535611 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:04:15.537451 (+ 1840us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:04:15.537997 (+ 546us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:04:15.538000 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:04:15.538000 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:04:15.538066 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:04:15.538275 (+ 209us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:04:15.538276 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:04:15.538276 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:04:15.538448 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:04:15.538625 (+ 177us) server_negotiation.cc:1035] Waiting for connection context
0923 15:04:15.538671 (+ 46us) server_negotiation.cc:299] Negotiation successful
0923 15:04:15.538757 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":98,"threads_started":1}
I20260923 15:04:15.539508 27138 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.75.189" port: 39335 } has no permanent_uuid. Determining permanent_uuid...
Sep 23 15:04:15 dist-test-slave-cnb0 krb5kdc[26907](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175855, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.188@KRBTEST.COM for kudu/127.13.75.189@KRBTEST.COM
Sep 23 15:04:15 dist-test-slave-cnb0 krb5kdc[26907](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175855, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.75.190@KRBTEST.COM
I20260923 15:04:15.547217 27067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:15.539876 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37170 (local address 127.13.75.189:39335)
0923 15:04:15.539916 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0923 15:04:15.539919 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:04:15.539928 (+ 9us) server_negotiation.cc:407] Connection header received
0923 15:04:15.539956 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:04:15.539957 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:04:15.539967 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:04:15.540014 (+ 47us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:04:15.540484 (+ 470us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:04:15.540960 (+ 476us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:04:15.541749 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:04:15.542041 (+ 292us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:04:15.542123 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:04:15.544419 (+ 2296us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:04:15.544421 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:04:15.544421 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:04:15.544428 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:04:15.546038 (+ 1610us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:04:15.546405 (+ 367us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:04:15.546406 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:04:15.546406 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:04:15.546466 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:04:15.546664 (+ 198us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:04:15.546665 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:04:15.546665 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:04:15.546786 (+ 121us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:04:15.546860 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0923 15:04:15.547047 (+ 187us) server_negotiation.cc:299] Negotiation successful
0923 15:04:15.547117 (+ 70us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260923 15:04:15.547217 27140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:15.539806 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.189:39335 (local address 127.0.0.1:37170)
0923 15:04:15.539850 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0923 15:04:15.539861 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 15:04:15.539905 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:04:15.540174 (+ 269us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:04:15.540175 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:04:15.540337 (+ 162us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:04:15.540401 (+ 64us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:04:15.540403 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:04:15.541037 (+ 634us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:04:15.541038 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:04:15.541668 (+ 630us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:04:15.541670 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:04:15.542136 (+ 466us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:04:15.542137 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:04:15.542215 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:04:15.542724 (+ 509us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:04:15.542730 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:04:15.544280 (+ 1550us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:04:15.546142 (+ 1862us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:04:15.546143 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:04:15.546144 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:04:15.546319 (+ 175us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:04:15.546531 (+ 212us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:04:15.546531 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:04:15.546532 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:04:15.546591 (+ 59us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:04:15.546856 (+ 265us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:04:15.546857 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:04:15.546929 (+ 72us) client_negotiation.cc:769] Sending connection context
0923 15:04:15.547020 (+ 91us) client_negotiation.cc:240] Negotiation successful
0923 15:04:15.547119 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":28}
I20260923 15:04:15.547735 27138 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.75.188" port: 45931 } has no permanent_uuid. Determining permanent_uuid...
I20260923 15:04:15.547794 26991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:15.540003 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50008 (local address 127.13.75.190:40393)
0923 15:04:15.540041 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0923 15:04:15.540044 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:04:15.540051 (+ 7us) server_negotiation.cc:407] Connection header received
0923 15:04:15.540072 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:04:15.540073 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:04:15.540081 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:04:15.540129 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:04:15.540719 (+ 590us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:04:15.541321 (+ 602us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:04:15.542134 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:04:15.542428 (+ 294us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:04:15.542523 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:04:15.544862 (+ 2339us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:04:15.544863 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:04:15.544863 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:04:15.544870 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:04:15.546336 (+ 1466us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:04:15.546770 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:04:15.546772 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:04:15.546772 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:04:15.546857 (+ 85us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:04:15.547123 (+ 266us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:04:15.547124 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:04:15.547125 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:04:15.547254 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:04:15.547481 (+ 227us) server_negotiation.cc:1035] Waiting for connection context
0923 15:04:15.547589 (+ 108us) server_negotiation.cc:299] Negotiation successful
0923 15:04:15.547680 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
Sep 23 15:04:15 dist-test-slave-cnb0 krb5kdc[26907](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175855, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.188@KRBTEST.COM for kudu/127.13.75.188@KRBTEST.COM
Sep 23 15:04:15 dist-test-slave-cnb0 krb5kdc[26907](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175855, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.75.189@KRBTEST.COM
I20260923 15:04:15.555764 27140 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:15.548082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.188:45931 (local address 127.0.0.1:40026)
0923 15:04:15.548160 (+ 78us) negotiation.cc:107] Waiting for socket to connect
0923 15:04:15.548172 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 15:04:15.548238 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:04:15.548497 (+ 259us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:04:15.548497 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:04:15.548634 (+ 137us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:04:15.548667 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:04:15.548668 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:04:15.549598 (+ 930us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:04:15.549599 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:04:15.550278 (+ 679us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:04:15.550280 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:04:15.550807 (+ 527us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:04:15.550808 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:04:15.550849 (+ 41us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:04:15.551386 (+ 537us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:04:15.551391 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:04:15.552636 (+ 1245us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:04:15.554497 (+ 1861us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:04:15.554498 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:04:15.554498 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:04:15.554667 (+ 169us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:04:15.554997 (+ 330us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:04:15.554998 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:04:15.554999 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:04:15.555069 (+ 70us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:04:15.555343 (+ 274us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:04:15.555344 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:04:15.555445 (+ 101us) client_negotiation.cc:769] Sending connection context
0923 15:04:15.555519 (+ 74us) client_negotiation.cc:240] Negotiation successful
0923 15:04:15.555636 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":61}
I20260923 15:04:15.555867 27142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:15.548117 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40026 (local address 127.13.75.188:45931)
0923 15:04:15.548164 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0923 15:04:15.548166 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:04:15.548222 (+ 56us) server_negotiation.cc:407] Connection header received
0923 15:04:15.548289 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:04:15.548290 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:04:15.548347 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:04:15.548408 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:04:15.548723 (+ 315us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:04:15.549511 (+ 788us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:04:15.550396 (+ 885us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:04:15.550736 (+ 340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:04:15.550810 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:04:15.552739 (+ 1929us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:04:15.552742 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:04:15.552742 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:04:15.552751 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:04:15.554397 (+ 1646us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:04:15.554754 (+ 357us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:04:15.554755 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:04:15.554756 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:04:15.554857 (+ 101us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:04:15.555155 (+ 298us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:04:15.555156 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:04:15.555156 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:04:15.555266 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:04:15.555360 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0923 15:04:15.555571 (+ 211us) server_negotiation.cc:299] Negotiation successful
0923 15:04:15.555705 (+ 134us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":43,"server-negotiator.queue_time_us":24}
I20260923 15:04:15.556353 26988 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.75.188" port: 45931 } attempt: 2
I20260923 15:04:15.557670 27138 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 97d3c821ddd447cf996a6fd286e6060b: Bootstrap starting.
I20260923 15:04:15.557862 27067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:15.548506 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37186 (local address 127.13.75.189:39335)
0923 15:04:15.548548 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0923 15:04:15.548550 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:04:15.548558 (+ 8us) server_negotiation.cc:407] Connection header received
0923 15:04:15.548583 (+ 25us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:04:15.548584 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:04:15.548593 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:04:15.548651 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:04:15.549317 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:04:15.550033 (+ 716us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:04:15.551141 (+ 1108us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:04:15.551371 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:04:15.551464 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:04:15.554613 (+ 3149us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:04:15.554615 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:04:15.554616 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:04:15.554624 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:04:15.556551 (+ 1927us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:04:15.556996 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:04:15.556997 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:04:15.556997 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:04:15.557059 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:04:15.557256 (+ 197us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:04:15.557257 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:04:15.557257 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:04:15.557377 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:04:15.557638 (+ 261us) server_negotiation.cc:1035] Waiting for connection context
0923 15:04:15.557678 (+ 40us) server_negotiation.cc:299] Negotiation successful
0923 15:04:15.557752 (+ 74us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260923 15:04:15.558780 27138 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 97d3c821ddd447cf996a6fd286e6060b: Neither blocks nor log segments found. Creating new log.
I20260923 15:04:15.559139 27138 log.cc:824] T 00000000000000000000000000000000 P 97d3c821ddd447cf996a6fd286e6060b: Log is configured to *not* fsync() on all Append() calls
I20260923 15:04:15.559791 13614 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 15:04:15.560598 27138 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 97d3c821ddd447cf996a6fd286e6060b: No bootstrap required, opened a new log
Sep 23 15:04:15 dist-test-slave-cnb0 krb5kdc[26907](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175855, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.190@KRBTEST.COM for kudu/127.13.75.188@KRBTEST.COM
I20260923 15:04:15.563235 27138 raft_consensus.cc:348] T 00000000000000000000000000000000 P 97d3c821ddd447cf996a6fd286e6060b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3dfecc8fcf041ce8e9fcda3eb907ada" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 40393 } } peers { permanent_uuid: "ee6df7ad5f004ac0a9680e393c0d9ff5" member_type: VOTER last_known_addr { host: "127.13.75.189" port: 39335 } } peers { permanent_uuid: "97d3c821ddd447cf996a6fd286e6060b" member_type: VOTER last_known_addr { host: "127.13.75.188" port: 45931 } }
I20260923 15:04:15.563540 27138 raft_consensus.cc:374] T 00000000000000000000000000000000 P 97d3c821ddd447cf996a6fd286e6060b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:04:15.563657 27138 raft_consensus.cc:729] T 00000000000000000000000000000000 P 97d3c821ddd447cf996a6fd286e6060b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 97d3c821ddd447cf996a6fd286e6060b, State: Initialized, Role: FOLLOWER
I20260923 15:04:15.563922 27138 consensus_queue.cc:260] T 00000000000000000000000000000000 P 97d3c821ddd447cf996a6fd286e6060b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3dfecc8fcf041ce8e9fcda3eb907ada" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 40393 } } peers { permanent_uuid: "ee6df7ad5f004ac0a9680e393c0d9ff5" member_type: VOTER last_known_addr { host: "127.13.75.189" port: 39335 } } peers { permanent_uuid: "97d3c821ddd447cf996a6fd286e6060b" member_type: VOTER last_known_addr { host: "127.13.75.188" port: 45931 } }
I20260923 15:04:15.564361 26990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:15.556617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.188:45931 (local address 127.0.0.1:40032)
0923 15:04:15.556671 (+ 54us) negotiation.cc:107] Waiting for socket to connect
0923 15:04:15.556681 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 15:04:15.556720 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:04:15.556895 (+ 175us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:04:15.556897 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:04:15.557063 (+ 166us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:04:15.557131 (+ 68us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:04:15.557133 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:04:15.557974 (+ 841us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:04:15.557976 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:04:15.558633 (+ 657us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:04:15.558635 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:04:15.559098 (+ 463us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:04:15.559099 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:04:15.559156 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:04:15.559565 (+ 409us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:04:15.559570 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:04:15.561004 (+ 1434us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:04:15.563045 (+ 2041us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:04:15.563047 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:04:15.563047 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:04:15.563303 (+ 256us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:04:15.563607 (+ 304us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:04:15.563608 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:04:15.563608 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:04:15.563680 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:04:15.564029 (+ 349us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:04:15.564030 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:04:15.564082 (+ 52us) client_negotiation.cc:769] Sending connection context
0923 15:04:15.564152 (+ 70us) client_negotiation.cc:240] Negotiation successful
0923 15:04:15.564263 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":35}
I20260923 15:04:15.564389 27138 sys_catalog.cc:564] T 00000000000000000000000000000000 P 97d3c821ddd447cf996a6fd286e6060b [sys.catalog]: configured and running, proceeding with master startup.
I20260923 15:04:15.564774 27142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:15.556647 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40032 (local address 127.13.75.188:45931)
0923 15:04:15.556796 (+ 149us) server_negotiation.cc:206] Beginning negotiation
0923 15:04:15.556797 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 15:04:15.556804 (+ 7us) server_negotiation.cc:407] Connection header received
0923 15:04:15.556821 (+ 17us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:04:15.556822 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:04:15.556831 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:04:15.556881 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:04:15.557200 (+ 319us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:04:15.557863 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:04:15.558732 (+ 869us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:04:15.559030 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:04:15.559098 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:04:15.561100 (+ 2002us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:04:15.561101 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:04:15.561102 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:04:15.561108 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:04:15.562813 (+ 1705us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:04:15.563453 (+ 640us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:04:15.563455 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:04:15.563455 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:04:15.563517 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:04:15.563757 (+ 240us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:04:15.563758 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:04:15.563759 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:04:15.563921 (+ 162us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:04:15.563990 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0923 15:04:15.564600 (+ 610us) server_negotiation.cc:299] Negotiation successful
0923 15:04:15.564687 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":130}
I20260923 15:04:15.567135 26988 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d3dfecc8fcf041ce8e9fcda3eb907ada: Bootstrap starting.
I20260923 15:04:15.567917 26988 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P d3dfecc8fcf041ce8e9fcda3eb907ada: Neither blocks nor log segments found. Creating new log.
I20260923 15:04:15.568238 26988 log.cc:824] T 00000000000000000000000000000000 P d3dfecc8fcf041ce8e9fcda3eb907ada: Log is configured to *not* fsync() on all Append() calls
W20260923 15:04:15.568648 27163 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 97d3c821ddd447cf996a6fd286e6060b: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260923 15:04:15.568713 27163 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260923 15:04:15.569032 27152 sys_catalog.cc:455] T 00000000000000000000000000000000 P 97d3c821ddd447cf996a6fd286e6060b [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3dfecc8fcf041ce8e9fcda3eb907ada" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 40393 } } peers { permanent_uuid: "ee6df7ad5f004ac0a9680e393c0d9ff5" member_type: VOTER last_known_addr { host: "127.13.75.189" port: 39335 } } peers { permanent_uuid: "97d3c821ddd447cf996a6fd286e6060b" member_type: VOTER last_known_addr { host: "127.13.75.188" port: 45931 } } }
I20260923 15:04:15.569159 27152 sys_catalog.cc:458] T 00000000000000000000000000000000 P 97d3c821ddd447cf996a6fd286e6060b [sys.catalog]: This master's current role is: FOLLOWER
I20260923 15:04:15.570863 27067 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:15.561486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37198 (local address 127.13.75.189:39335)
0923 15:04:15.561526 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0923 15:04:15.561528 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:04:15.562001 (+ 473us) server_negotiation.cc:407] Connection header received
0923 15:04:15.562095 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:04:15.562096 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:04:15.562106 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:04:15.562138 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:04:15.562934 (+ 796us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:04:15.563612 (+ 678us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:04:15.564647 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:04:15.564930 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:04:15.564967 (+ 37us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:04:15.566510 (+ 1543us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:04:15.566512 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:04:15.566513 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:04:15.566520 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:04:15.568409 (+ 1889us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:04:15.569907 (+ 1498us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:04:15.569908 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:04:15.569908 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:04:15.569965 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:04:15.570183 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:04:15.570184 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:04:15.570184 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:04:15.570326 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:04:15.570421 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0923 15:04:15.570666 (+ 245us) server_negotiation.cc:299] Negotiation successful
0923 15:04:15.570744 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260923 15:04:15.572515 26991 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:15.561734 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50024 (local address 127.13.75.190:40393)
0923 15:04:15.561794 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0923 15:04:15.561796 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:04:15.561804 (+ 8us) server_negotiation.cc:407] Connection header received
0923 15:04:15.561825 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:04:15.561826 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:04:15.561837 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:04:15.561888 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:04:15.562682 (+ 794us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:04:15.563460 (+ 778us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:04:15.564512 (+ 1052us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:04:15.564807 (+ 295us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:04:15.564845 (+ 38us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:04:15.566525 (+ 1680us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:04:15.566526 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:04:15.566527 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:04:15.566533 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:04:15.570562 (+ 4029us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:04:15.571620 (+ 1058us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:04:15.571621 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:04:15.571621 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:04:15.571689 (+ 68us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:04:15.571907 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:04:15.571908 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:04:15.571909 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:04:15.572031 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:04:15.572107 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0923 15:04:15.572329 (+ 222us) server_negotiation.cc:299] Negotiation successful
0923 15:04:15.572410 (+ 81us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260923 15:04:15.573128 27142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:15.561788 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40046 (local address 127.13.75.188:45931)
0923 15:04:15.564842 (+ 3054us) server_negotiation.cc:206] Beginning negotiation
0923 15:04:15.564844 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:04:15.564855 (+ 11us) server_negotiation.cc:407] Connection header received
0923 15:04:15.564879 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:04:15.564880 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:04:15.564893 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:04:15.564952 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:04:15.565983 (+ 1031us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:04:15.566738 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:04:15.567834 (+ 1096us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:04:15.568144 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:04:15.568197 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:04:15.569821 (+ 1624us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:04:15.569823 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:04:15.569823 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:04:15.569830 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:04:15.571676 (+ 1846us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:04:15.572144 (+ 468us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:04:15.572145 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:04:15.572146 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:04:15.572202 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:04:15.572444 (+ 242us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:04:15.572445 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:04:15.572446 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:04:15.572578 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:04:15.572659 (+ 81us) server_negotiation.cc:1035] Waiting for connection context
0923 15:04:15.572949 (+ 290us) server_negotiation.cc:299] Negotiation successful
0923 15:04:15.573029 (+ 80us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3037,"thread_start_us":64,"threads_started":1}
I20260923 15:04:15.569629 27070 master_runner.cc:186] Error getting master registration for 127.13.75.190:40393: OK, instance_id { permanent_uuid: "d3dfecc8fcf041ce8e9fcda3eb907ada" instance_seqno: 1790175855213777 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260923 15:04:15.573347 27070 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260923 15:04:15.578337 26988 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P d3dfecc8fcf041ce8e9fcda3eb907ada: No bootstrap required, opened a new log
I20260923 15:04:15.579967 26988 raft_consensus.cc:348] T 00000000000000000000000000000000 P d3dfecc8fcf041ce8e9fcda3eb907ada [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3dfecc8fcf041ce8e9fcda3eb907ada" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 40393 } } peers { permanent_uuid: "ee6df7ad5f004ac0a9680e393c0d9ff5" member_type: VOTER last_known_addr { host: "127.13.75.189" port: 39335 } } peers { permanent_uuid: "97d3c821ddd447cf996a6fd286e6060b" member_type: VOTER last_known_addr { host: "127.13.75.188" port: 45931 } }
I20260923 15:04:15.580080 26988 raft_consensus.cc:374] T 00000000000000000000000000000000 P d3dfecc8fcf041ce8e9fcda3eb907ada [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:04:15.580101 26988 raft_consensus.cc:729] T 00000000000000000000000000000000 P d3dfecc8fcf041ce8e9fcda3eb907ada [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d3dfecc8fcf041ce8e9fcda3eb907ada, State: Initialized, Role: FOLLOWER
I20260923 15:04:15.580222 26988 consensus_queue.cc:260] T 00000000000000000000000000000000 P d3dfecc8fcf041ce8e9fcda3eb907ada [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3dfecc8fcf041ce8e9fcda3eb907ada" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 40393 } } peers { permanent_uuid: "ee6df7ad5f004ac0a9680e393c0d9ff5" member_type: VOTER last_known_addr { host: "127.13.75.189" port: 39335 } } peers { permanent_uuid: "97d3c821ddd447cf996a6fd286e6060b" member_type: VOTER last_known_addr { host: "127.13.75.188" port: 45931 } }
I20260923 15:04:15.580524 27166 sys_catalog.cc:455] T 00000000000000000000000000000000 P d3dfecc8fcf041ce8e9fcda3eb907ada [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3dfecc8fcf041ce8e9fcda3eb907ada" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 40393 } } peers { permanent_uuid: "ee6df7ad5f004ac0a9680e393c0d9ff5" member_type: VOTER last_known_addr { host: "127.13.75.189" port: 39335 } } peers { permanent_uuid: "97d3c821ddd447cf996a6fd286e6060b" member_type: VOTER last_known_addr { host: "127.13.75.188" port: 45931 } } }
I20260923 15:04:15.580600 26988 sys_catalog.cc:564] T 00000000000000000000000000000000 P d3dfecc8fcf041ce8e9fcda3eb907ada [sys.catalog]: configured and running, proceeding with master startup.
I20260923 15:04:15.580605 27166 sys_catalog.cc:458] T 00000000000000000000000000000000 P d3dfecc8fcf041ce8e9fcda3eb907ada [sys.catalog]: This master's current role is: FOLLOWER
W20260923 15:04:15.582875 27177 catalog_manager.cc:1558] T 00000000000000000000000000000000 P d3dfecc8fcf041ce8e9fcda3eb907ada: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260923 15:04:15.582983 27177 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260923 15:04:15.583642 26920 master_runner.cc:186] Error getting master registration for 127.13.75.189:39335: OK, instance_id { permanent_uuid: "ee6df7ad5f004ac0a9680e393c0d9ff5" instance_seqno: 1790175855364777 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260923 15:04:15.586584 26920 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260923 15:04:15.600409 27063 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.75.188" port: 45931 } attempt: 3
Sep 23 15:04:15 dist-test-slave-cnb0 krb5kdc[26907](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175855, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.189@KRBTEST.COM for kudu/127.13.75.188@KRBTEST.COM
I20260923 15:04:15.609378 27065 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:15.600762 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.188:45931 (local address 127.0.0.1:40050)
0923 15:04:15.600819 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0923 15:04:15.600831 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 15:04:15.600872 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:04:15.601066 (+ 194us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:04:15.601067 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:04:15.601260 (+ 193us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:04:15.601315 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:04:15.601317 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:04:15.602416 (+ 1099us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:04:15.602418 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:04:15.603251 (+ 833us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:04:15.603254 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:04:15.603749 (+ 495us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:04:15.603751 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:04:15.603812 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:04:15.604355 (+ 543us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:04:15.604361 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:04:15.606083 (+ 1722us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:04:15.608172 (+ 2089us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:04:15.608174 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:04:15.608175 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:04:15.608400 (+ 225us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:04:15.608623 (+ 223us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:04:15.608624 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:04:15.608624 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:04:15.608693 (+ 69us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:04:15.609026 (+ 333us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:04:15.609027 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:04:15.609078 (+ 51us) client_negotiation.cc:769] Sending connection context
0923 15:04:15.609150 (+ 72us) client_negotiation.cc:240] Negotiation successful
0923 15:04:15.609259 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":37}
I20260923 15:04:15.609391 27142 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:15.600838 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40050 (local address 127.13.75.188:45931)
0923 15:04:15.600897 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0923 15:04:15.600899 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:04:15.600932 (+ 33us) server_negotiation.cc:407] Connection header received
0923 15:04:15.600959 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:04:15.600960 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:04:15.600971 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:04:15.601046 (+ 75us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:04:15.601405 (+ 359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:04:15.602253 (+ 848us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:04:15.603351 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:04:15.603662 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:04:15.603741 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:04:15.606174 (+ 2433us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:04:15.606175 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:04:15.606176 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:04:15.606183 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:04:15.608075 (+ 1892us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:04:15.608484 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:04:15.608485 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:04:15.608486 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:04:15.608547 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:04:15.608762 (+ 215us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:04:15.608763 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:04:15.608763 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:04:15.608905 (+ 142us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:04:15.608993 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0923 15:04:15.609207 (+ 214us) server_negotiation.cc:299] Negotiation successful
0923 15:04:15.609294 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260923 15:04:15.611721 27063 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ee6df7ad5f004ac0a9680e393c0d9ff5: Bootstrap starting.
I20260923 15:04:15.612476 27063 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ee6df7ad5f004ac0a9680e393c0d9ff5: Neither blocks nor log segments found. Creating new log.
I20260923 15:04:15.612771 27063 log.cc:824] T 00000000000000000000000000000000 P ee6df7ad5f004ac0a9680e393c0d9ff5: Log is configured to *not* fsync() on all Append() calls
I20260923 15:04:15.613667 27063 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ee6df7ad5f004ac0a9680e393c0d9ff5: No bootstrap required, opened a new log
I20260923 15:04:15.615733 27063 raft_consensus.cc:348] T 00000000000000000000000000000000 P ee6df7ad5f004ac0a9680e393c0d9ff5 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3dfecc8fcf041ce8e9fcda3eb907ada" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 40393 } } peers { permanent_uuid: "ee6df7ad5f004ac0a9680e393c0d9ff5" member_type: VOTER last_known_addr { host: "127.13.75.189" port: 39335 } } peers { permanent_uuid: "97d3c821ddd447cf996a6fd286e6060b" member_type: VOTER last_known_addr { host: "127.13.75.188" port: 45931 } }
I20260923 15:04:15.615881 27063 raft_consensus.cc:374] T 00000000000000000000000000000000 P ee6df7ad5f004ac0a9680e393c0d9ff5 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:04:15.615918 27063 raft_consensus.cc:729] T 00000000000000000000000000000000 P ee6df7ad5f004ac0a9680e393c0d9ff5 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ee6df7ad5f004ac0a9680e393c0d9ff5, State: Initialized, Role: FOLLOWER
I20260923 15:04:15.616042 27063 consensus_queue.cc:260] T 00000000000000000000000000000000 P ee6df7ad5f004ac0a9680e393c0d9ff5 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3dfecc8fcf041ce8e9fcda3eb907ada" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 40393 } } peers { permanent_uuid: "ee6df7ad5f004ac0a9680e393c0d9ff5" member_type: VOTER last_known_addr { host: "127.13.75.189" port: 39335 } } peers { permanent_uuid: "97d3c821ddd447cf996a6fd286e6060b" member_type: VOTER last_known_addr { host: "127.13.75.188" port: 45931 } }
I20260923 15:04:15.616329 27180 sys_catalog.cc:455] T 00000000000000000000000000000000 P ee6df7ad5f004ac0a9680e393c0d9ff5 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3dfecc8fcf041ce8e9fcda3eb907ada" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 40393 } } peers { permanent_uuid: "ee6df7ad5f004ac0a9680e393c0d9ff5" member_type: VOTER last_known_addr { host: "127.13.75.189" port: 39335 } } peers { permanent_uuid: "97d3c821ddd447cf996a6fd286e6060b" member_type: VOTER last_known_addr { host: "127.13.75.188" port: 45931 } } }
I20260923 15:04:15.616384 27063 sys_catalog.cc:564] T 00000000000000000000000000000000 P ee6df7ad5f004ac0a9680e393c0d9ff5 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 15:04:15.616464 27180 sys_catalog.cc:458] T 00000000000000000000000000000000 P ee6df7ad5f004ac0a9680e393c0d9ff5 [sys.catalog]: This master's current role is: FOLLOWER
W20260923 15:04:15.618418 27191 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ee6df7ad5f004ac0a9680e393c0d9ff5: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260923 15:04:15.618533 27191 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260923 15:04:15.620477 26995 master_runner.cc:263] No leader master found from master ee6df7ad5f004ac0a9680e393c0d9ff5
I20260923 15:04:15.620533 26995 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
W20260923 15:04:15.711886 26941 tablet.cc:2369] T 00000000000000000000000000000000 P d3dfecc8fcf041ce8e9fcda3eb907ada: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260923 15:04:15.770094 27091 tablet.cc:2369] T 00000000000000000000000000000000 P 97d3c821ddd447cf996a6fd286e6060b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260923 15:04:15.862691 27016 tablet.cc:2369] T 00000000000000000000000000000000 P ee6df7ad5f004ac0a9680e393c0d9ff5: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 15:04:16.023913 27152 raft_consensus.cc:482] T 00000000000000000000000000000000 P 97d3c821ddd447cf996a6fd286e6060b [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 15:04:16.024065 27152 raft_consensus.cc:504] T 00000000000000000000000000000000 P 97d3c821ddd447cf996a6fd286e6060b [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3dfecc8fcf041ce8e9fcda3eb907ada" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 40393 } } peers { permanent_uuid: "ee6df7ad5f004ac0a9680e393c0d9ff5" member_type: VOTER last_known_addr { host: "127.13.75.189" port: 39335 } } peers { permanent_uuid: "97d3c821ddd447cf996a6fd286e6060b" member_type: VOTER last_known_addr { host: "127.13.75.188" port: 45931 } }
I20260923 15:04:16.024376 27152 leader_election.cc:288] T 00000000000000000000000000000000 P 97d3c821ddd447cf996a6fd286e6060b [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d3dfecc8fcf041ce8e9fcda3eb907ada (127.13.75.190:40393), ee6df7ad5f004ac0a9680e393c0d9ff5 (127.13.75.189:39335)
I20260923 15:04:16.024804 26962 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "97d3c821ddd447cf996a6fd286e6060b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3dfecc8fcf041ce8e9fcda3eb907ada" is_pre_election: true
I20260923 15:04:16.024967 26962 raft_consensus.cc:2454] T 00000000000000000000000000000000 P d3dfecc8fcf041ce8e9fcda3eb907ada [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 97d3c821ddd447cf996a6fd286e6060b in term 0.
I20260923 15:04:16.024892 27037 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "97d3c821ddd447cf996a6fd286e6060b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee6df7ad5f004ac0a9680e393c0d9ff5" is_pre_election: true
I20260923 15:04:16.025095 27037 raft_consensus.cc:2454] T 00000000000000000000000000000000 P ee6df7ad5f004ac0a9680e393c0d9ff5 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 97d3c821ddd447cf996a6fd286e6060b in term 0.
I20260923 15:04:16.025179 27086 leader_election.cc:302] T 00000000000000000000000000000000 P 97d3c821ddd447cf996a6fd286e6060b [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 97d3c821ddd447cf996a6fd286e6060b, d3dfecc8fcf041ce8e9fcda3eb907ada; no voters:
I20260923 15:04:16.025357 27152 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 97d3c821ddd447cf996a6fd286e6060b [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 15:04:16.025452 27152 raft_consensus.cc:482] T 00000000000000000000000000000000 P 97d3c821ddd447cf996a6fd286e6060b [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 15:04:16.025490 27152 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 97d3c821ddd447cf996a6fd286e6060b [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:04:16.026242 27152 raft_consensus.cc:504] T 00000000000000000000000000000000 P 97d3c821ddd447cf996a6fd286e6060b [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3dfecc8fcf041ce8e9fcda3eb907ada" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 40393 } } peers { permanent_uuid: "ee6df7ad5f004ac0a9680e393c0d9ff5" member_type: VOTER last_known_addr { host: "127.13.75.189" port: 39335 } } peers { permanent_uuid: "97d3c821ddd447cf996a6fd286e6060b" member_type: VOTER last_known_addr { host: "127.13.75.188" port: 45931 } }
I20260923 15:04:16.026393 27152 leader_election.cc:288] T 00000000000000000000000000000000 P 97d3c821ddd447cf996a6fd286e6060b [CANDIDATE]: Term 1 election: Requested vote from peers d3dfecc8fcf041ce8e9fcda3eb907ada (127.13.75.190:40393), ee6df7ad5f004ac0a9680e393c0d9ff5 (127.13.75.189:39335)
I20260923 15:04:16.026633 27037 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "97d3c821ddd447cf996a6fd286e6060b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "ee6df7ad5f004ac0a9680e393c0d9ff5"
I20260923 15:04:16.026604 26962 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "97d3c821ddd447cf996a6fd286e6060b" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d3dfecc8fcf041ce8e9fcda3eb907ada"
I20260923 15:04:16.026701 26962 raft_consensus.cc:3037] T 00000000000000000000000000000000 P d3dfecc8fcf041ce8e9fcda3eb907ada [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:04:16.026721 27037 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ee6df7ad5f004ac0a9680e393c0d9ff5 [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:04:16.027657 26962 raft_consensus.cc:2454] T 00000000000000000000000000000000 P d3dfecc8fcf041ce8e9fcda3eb907ada [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 97d3c821ddd447cf996a6fd286e6060b in term 1.
I20260923 15:04:16.027724 27037 raft_consensus.cc:2454] T 00000000000000000000000000000000 P ee6df7ad5f004ac0a9680e393c0d9ff5 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 97d3c821ddd447cf996a6fd286e6060b in term 1.
I20260923 15:04:16.027904 27086 leader_election.cc:302] T 00000000000000000000000000000000 P 97d3c821ddd447cf996a6fd286e6060b [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 97d3c821ddd447cf996a6fd286e6060b, ee6df7ad5f004ac0a9680e393c0d9ff5; no voters:
I20260923 15:04:16.027999 27152 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 97d3c821ddd447cf996a6fd286e6060b [term 1 FOLLOWER]: Leader election won for term 1
I20260923 15:04:16.028143 27152 raft_consensus.cc:686] T 00000000000000000000000000000000 P 97d3c821ddd447cf996a6fd286e6060b [term 1 LEADER]: Becoming Leader. State: Replica: 97d3c821ddd447cf996a6fd286e6060b, State: Running, Role: LEADER
I20260923 15:04:16.028254 27152 consensus_queue.cc:237] T 00000000000000000000000000000000 P 97d3c821ddd447cf996a6fd286e6060b [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3dfecc8fcf041ce8e9fcda3eb907ada" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 40393 } } peers { permanent_uuid: "ee6df7ad5f004ac0a9680e393c0d9ff5" member_type: VOTER last_known_addr { host: "127.13.75.189" port: 39335 } } peers { permanent_uuid: "97d3c821ddd447cf996a6fd286e6060b" member_type: VOTER last_known_addr { host: "127.13.75.188" port: 45931 } }
I20260923 15:04:16.028697 27193 sys_catalog.cc:455] T 00000000000000000000000000000000 P 97d3c821ddd447cf996a6fd286e6060b [sys.catalog]: SysCatalogTable state changed. Reason: New leader 97d3c821ddd447cf996a6fd286e6060b. Latest consensus state: current_term: 1 leader_uuid: "97d3c821ddd447cf996a6fd286e6060b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3dfecc8fcf041ce8e9fcda3eb907ada" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 40393 } } peers { permanent_uuid: "ee6df7ad5f004ac0a9680e393c0d9ff5" member_type: VOTER last_known_addr { host: "127.13.75.189" port: 39335 } } peers { permanent_uuid: "97d3c821ddd447cf996a6fd286e6060b" member_type: VOTER last_known_addr { host: "127.13.75.188" port: 45931 } } }
I20260923 15:04:16.028798 27193 sys_catalog.cc:458] T 00000000000000000000000000000000 P 97d3c821ddd447cf996a6fd286e6060b [sys.catalog]: This master's current role is: LEADER
I20260923 15:04:16.029002 27195 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 15:04:16.029196 27195 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 15:04:16.030665 27037 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ee6df7ad5f004ac0a9680e393c0d9ff5 [term 1 FOLLOWER]: Refusing update from remote peer 97d3c821ddd447cf996a6fd286e6060b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 15:04:16.030665 26962 raft_consensus.cc:1261] T 00000000000000000000000000000000 P d3dfecc8fcf041ce8e9fcda3eb907ada [term 1 FOLLOWER]: Refusing update from remote peer 97d3c821ddd447cf996a6fd286e6060b: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 15:04:16.031055 27152 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 97d3c821ddd447cf996a6fd286e6060b [LEADER]: Connected to new peer: Peer: permanent_uuid: "d3dfecc8fcf041ce8e9fcda3eb907ada" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 40393 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 15:04:16.031155 27193 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 97d3c821ddd447cf996a6fd286e6060b [LEADER]: Connected to new peer: Peer: permanent_uuid: "ee6df7ad5f004ac0a9680e393c0d9ff5" member_type: VOTER last_known_addr { host: "127.13.75.189" port: 39335 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 15:04:16.032335 27180 sys_catalog.cc:455] T 00000000000000000000000000000000 P ee6df7ad5f004ac0a9680e393c0d9ff5 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 97d3c821ddd447cf996a6fd286e6060b. Latest consensus state: current_term: 1 leader_uuid: "97d3c821ddd447cf996a6fd286e6060b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3dfecc8fcf041ce8e9fcda3eb907ada" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 40393 } } peers { permanent_uuid: "ee6df7ad5f004ac0a9680e393c0d9ff5" member_type: VOTER last_known_addr { host: "127.13.75.189" port: 39335 } } peers { permanent_uuid: "97d3c821ddd447cf996a6fd286e6060b" member_type: VOTER last_known_addr { host: "127.13.75.188" port: 45931 } } }
I20260923 15:04:16.032378 27166 sys_catalog.cc:455] T 00000000000000000000000000000000 P d3dfecc8fcf041ce8e9fcda3eb907ada [sys.catalog]: SysCatalogTable state changed. Reason: New leader 97d3c821ddd447cf996a6fd286e6060b. Latest consensus state: current_term: 1 leader_uuid: "97d3c821ddd447cf996a6fd286e6060b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3dfecc8fcf041ce8e9fcda3eb907ada" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 40393 } } peers { permanent_uuid: "ee6df7ad5f004ac0a9680e393c0d9ff5" member_type: VOTER last_known_addr { host: "127.13.75.189" port: 39335 } } peers { permanent_uuid: "97d3c821ddd447cf996a6fd286e6060b" member_type: VOTER last_known_addr { host: "127.13.75.188" port: 45931 } } }
I20260923 15:04:16.032436 27180 sys_catalog.cc:458] T 00000000000000000000000000000000 P ee6df7ad5f004ac0a9680e393c0d9ff5 [sys.catalog]: This master's current role is: FOLLOWER
I20260923 15:04:16.032459 27166 sys_catalog.cc:458] T 00000000000000000000000000000000 P d3dfecc8fcf041ce8e9fcda3eb907ada [sys.catalog]: This master's current role is: FOLLOWER
I20260923 15:04:16.033381 27193 sys_catalog.cc:455] T 00000000000000000000000000000000 P 97d3c821ddd447cf996a6fd286e6060b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "97d3c821ddd447cf996a6fd286e6060b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3dfecc8fcf041ce8e9fcda3eb907ada" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 40393 } } peers { permanent_uuid: "ee6df7ad5f004ac0a9680e393c0d9ff5" member_type: VOTER last_known_addr { host: "127.13.75.189" port: 39335 } } peers { permanent_uuid: "97d3c821ddd447cf996a6fd286e6060b" member_type: VOTER last_known_addr { host: "127.13.75.188" port: 45931 } } }
I20260923 15:04:16.033499 27193 sys_catalog.cc:458] T 00000000000000000000000000000000 P 97d3c821ddd447cf996a6fd286e6060b [sys.catalog]: This master's current role is: LEADER
I20260923 15:04:16.033579 27180 sys_catalog.cc:455] T 00000000000000000000000000000000 P ee6df7ad5f004ac0a9680e393c0d9ff5 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "97d3c821ddd447cf996a6fd286e6060b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3dfecc8fcf041ce8e9fcda3eb907ada" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 40393 } } peers { permanent_uuid: "ee6df7ad5f004ac0a9680e393c0d9ff5" member_type: VOTER last_known_addr { host: "127.13.75.189" port: 39335 } } peers { permanent_uuid: "97d3c821ddd447cf996a6fd286e6060b" member_type: VOTER last_known_addr { host: "127.13.75.188" port: 45931 } } }
I20260923 15:04:16.033653 27180 sys_catalog.cc:458] T 00000000000000000000000000000000 P ee6df7ad5f004ac0a9680e393c0d9ff5 [sys.catalog]: This master's current role is: FOLLOWER
I20260923 15:04:16.033720 27195 catalog_manager.cc:1347] Generated new cluster ID: cef6c262b15a489eb359377b9d4e7240
I20260923 15:04:16.033753 27195 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 15:04:16.033949 27193 sys_catalog.cc:455] T 00000000000000000000000000000000 P 97d3c821ddd447cf996a6fd286e6060b [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "97d3c821ddd447cf996a6fd286e6060b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3dfecc8fcf041ce8e9fcda3eb907ada" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 40393 } } peers { permanent_uuid: "ee6df7ad5f004ac0a9680e393c0d9ff5" member_type: VOTER last_known_addr { host: "127.13.75.189" port: 39335 } } peers { permanent_uuid: "97d3c821ddd447cf996a6fd286e6060b" member_type: VOTER last_known_addr { host: "127.13.75.188" port: 45931 } } }
I20260923 15:04:16.034036 27193 sys_catalog.cc:458] T 00000000000000000000000000000000 P 97d3c821ddd447cf996a6fd286e6060b [sys.catalog]: This master's current role is: LEADER
I20260923 15:04:16.034300 27166 sys_catalog.cc:455] T 00000000000000000000000000000000 P d3dfecc8fcf041ce8e9fcda3eb907ada [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "97d3c821ddd447cf996a6fd286e6060b" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "d3dfecc8fcf041ce8e9fcda3eb907ada" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 40393 } } peers { permanent_uuid: "ee6df7ad5f004ac0a9680e393c0d9ff5" member_type: VOTER last_known_addr { host: "127.13.75.189" port: 39335 } } peers { permanent_uuid: "97d3c821ddd447cf996a6fd286e6060b" member_type: VOTER last_known_addr { host: "127.13.75.188" port: 45931 } } }
I20260923 15:04:16.034394 27166 sys_catalog.cc:458] T 00000000000000000000000000000000 P d3dfecc8fcf041ce8e9fcda3eb907ada [sys.catalog]: This master's current role is: FOLLOWER
I20260923 15:04:16.052047 27195 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 15:04:16.052742 27195 catalog_manager.cc:1504] Loading token signing keys...
I20260923 15:04:16.058429 27195 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 97d3c821ddd447cf996a6fd286e6060b: Generated new TSK 0
I20260923 15:04:16.058687 27195 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 15:04:16.583434 27177 catalog_manager.cc:1259] Loaded cluster ID: cef6c262b15a489eb359377b9d4e7240
I20260923 15:04:16.583513 27177 catalog_manager.cc:1552] T 00000000000000000000000000000000 P d3dfecc8fcf041ce8e9fcda3eb907ada: loading cluster ID for follower catalog manager: success
I20260923 15:04:16.584283 27177 catalog_manager.cc:1574] T 00000000000000000000000000000000 P d3dfecc8fcf041ce8e9fcda3eb907ada: acquiring CA information for follower catalog manager: success
I20260923 15:04:16.584537 27177 catalog_manager.cc:1602] T 00000000000000000000000000000000 P d3dfecc8fcf041ce8e9fcda3eb907ada: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260923 15:04:16.619019 27191 catalog_manager.cc:1259] Loaded cluster ID: cef6c262b15a489eb359377b9d4e7240
I20260923 15:04:16.619094 27191 catalog_manager.cc:1552] T 00000000000000000000000000000000 P ee6df7ad5f004ac0a9680e393c0d9ff5: loading cluster ID for follower catalog manager: success
I20260923 15:04:16.620002 27191 catalog_manager.cc:1574] T 00000000000000000000000000000000 P ee6df7ad5f004ac0a9680e393c0d9ff5: acquiring CA information for follower catalog manager: success
I20260923 15:04:16.620355 27191 catalog_manager.cc:1602] T 00000000000000000000000000000000 P ee6df7ad5f004ac0a9680e393c0d9ff5: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260923 15:04:17.112738 13614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKC4soy/build/release/bin/kudu with pid 26920
I20260923 15:04:17.118546 13614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKC4soy/build/release/bin/kudu with pid 26995
I20260923 15:04:17.124429 13614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKC4soy/build/release/bin/kudu with pid 27070
2026-09-23T15:04:17Z chronyd exiting
[ OK ] SecurityITest.IPKICACert (3238 ms)
[ RUN ] SecurityITest.IPKICACertWebServerEndPoints
Loading random data
Initializing database '/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790175691491353-13614-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 15:04:17 dist-test-slave-cnb0 krb5kdc[27210](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 15:04:17 dist-test-slave-cnb0 krb5kdc[27210](info): set up 2 sockets
Sep 23 15:04:17 dist-test-slave-cnb0 krb5kdc[27210](info): commencing operation
krb5kdc: starting...
W20260923 15:04:18.180226 13614 mini_kdc.cc:121] Time spent starting KDC: real 1.035s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790175691491353-13614-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790175691491353-13614-0/krb5kdc/test-user.keytab.
Sep 23 15:04:18 dist-test-slave-cnb0 krb5kdc[27210](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175858, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T15:04:18Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T15:04:18Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.75.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.75.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.75.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.190.keytab.
Entry for principal kudu/127.13.75.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.190.keytab.
I20260923 15:04:18.306355 13614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKC4soy/build/release/bin/kudu
/tmp/dist-test-taskKC4soy/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790175691491353-13614-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790175691491353-13614-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790175691491353-13614-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790175691491353-13614-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.75.190:46395
--webserver_interface=127.13.75.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.190.keytab
--principal=kudu/127.13.75.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.75.148:46541
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.75.190:46395
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790175691491353-13614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790175691491353-13614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790175691491353-13614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:04:18.387971 27223 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:04:18.388195 27223 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:04:18.388219 27223 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:04:18.389361 27223 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 15:04:18.389398 27223 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:04:18.389412 27223 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:04:18.389425 27223 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 15:04:18.389437 27223 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 15:04:18.390926 27223 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.75.148:46541
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790175691491353-13614-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790175691491353-13614-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.75.190:46395
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.190.keytab
--principal=kudu/127.13.75.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.75.190:46395
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790175691491353-13614-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.75.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27223
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790175691491353-13614-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:04:18.391149 27223 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:04:18.391335 27223 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:04:18.396498 27231 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:04:18.396526 27229 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:04:18.396723 27223 server_base.cc:1034] running on GCE node
W20260923 15:04:18.396515 27228 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:04:18.397135 27223 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:04:18.397500 27223 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:04:18.398696 27223 hybrid_clock.cc:648] HybridClock initialized: now 1790175858398682 us; error 40 us; skew 500 ppm
Sep 23 15:04:18 dist-test-slave-cnb0 krb5kdc[27210](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175858, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:04:18.401428 27223 init.cc:377] Logged in from keytab as kudu/127.13.75.190@KRBTEST.COM (short username kudu)
I20260923 15:04:18.401893 27223 webserver.cc:460] Webserver started at http://127.13.75.190:41269/ using document root <none> and password file <none>
I20260923 15:04:18.402094 27223 fs_manager.cc:359] Metadata directory not provided
I20260923 15:04:18.402135 27223 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:04:18.402225 27223 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:04:18.403371 27223 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790175691491353-13614-0/minicluster-data/master-0/data/instance:
uuid: "16f7d668b467495b83b117535850e625"
format_stamp: "Formatted at 2026-09-23 15:04:18 on dist-test-slave-cnb0"
server_key: "363fc8dc6eafc0bd030f1110623bc67f"
server_key_iv: "04490bff69423c40bacb8af0f71bcf19"
server_key_version: "encryptionkey@0"
I20260923 15:04:18.403693 27223 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790175691491353-13614-0/minicluster-data/master-0/wal/instance:
uuid: "16f7d668b467495b83b117535850e625"
format_stamp: "Formatted at 2026-09-23 15:04:18 on dist-test-slave-cnb0"
server_key: "363fc8dc6eafc0bd030f1110623bc67f"
server_key_iv: "04490bff69423c40bacb8af0f71bcf19"
server_key_version: "encryptionkey@0"
I20260923 15:04:18.405057 27223 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 15:04:18.405843 27238 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:04:18.406071 27223 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:04:18.406154 27223 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790175691491353-13614-0/minicluster-data/master-0/data,/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790175691491353-13614-0/minicluster-data/master-0/wal
uuid: "16f7d668b467495b83b117535850e625"
format_stamp: "Formatted at 2026-09-23 15:04:18 on dist-test-slave-cnb0"
server_key: "363fc8dc6eafc0bd030f1110623bc67f"
server_key_iv: "04490bff69423c40bacb8af0f71bcf19"
server_key_version: "encryptionkey@0"
I20260923 15:04:18.406211 27223 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790175691491353-13614-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790175691491353-13614-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790175691491353-13614-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:04:18.424399 27223 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:04:18.426718 27223 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:04:18.426833 27223 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:04:18.431217 27223 rpc_server.cc:307] RPC server started. Bound to: 127.13.75.190:46395
I20260923 15:04:18.431264 27290 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.75.190:46395 every 8 connection(s)
I20260923 15:04:18.431696 27223 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790175691491353-13614-0/minicluster-data/master-0/data/info.pb
I20260923 15:04:18.432662 27291 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:04:18.434998 27291 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 16f7d668b467495b83b117535850e625: Bootstrap starting.
I20260923 15:04:18.435549 27291 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 16f7d668b467495b83b117535850e625: Neither blocks nor log segments found. Creating new log.
I20260923 15:04:18.435783 27291 log.cc:824] T 00000000000000000000000000000000 P 16f7d668b467495b83b117535850e625: Log is configured to *not* fsync() on all Append() calls
I20260923 15:04:18.436560 27291 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 16f7d668b467495b83b117535850e625: No bootstrap required, opened a new log
I20260923 15:04:18.438135 27291 raft_consensus.cc:348] T 00000000000000000000000000000000 P 16f7d668b467495b83b117535850e625 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16f7d668b467495b83b117535850e625" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 46395 } }
I20260923 15:04:18.438275 27291 raft_consensus.cc:374] T 00000000000000000000000000000000 P 16f7d668b467495b83b117535850e625 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:04:18.438298 27291 raft_consensus.cc:729] T 00000000000000000000000000000000 P 16f7d668b467495b83b117535850e625 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 16f7d668b467495b83b117535850e625, State: Initialized, Role: FOLLOWER
I20260923 15:04:18.438441 27291 consensus_queue.cc:260] T 00000000000000000000000000000000 P 16f7d668b467495b83b117535850e625 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16f7d668b467495b83b117535850e625" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 46395 } }
I20260923 15:04:18.438526 27291 raft_consensus.cc:388] T 00000000000000000000000000000000 P 16f7d668b467495b83b117535850e625 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 15:04:18.438550 27291 raft_consensus.cc:482] T 00000000000000000000000000000000 P 16f7d668b467495b83b117535850e625 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 15:04:18.438580 27291 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 16f7d668b467495b83b117535850e625 [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:04:18.439154 27291 raft_consensus.cc:504] T 00000000000000000000000000000000 P 16f7d668b467495b83b117535850e625 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16f7d668b467495b83b117535850e625" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 46395 } }
I20260923 15:04:18.439239 27291 leader_election.cc:302] T 00000000000000000000000000000000 P 16f7d668b467495b83b117535850e625 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 16f7d668b467495b83b117535850e625; no voters:
I20260923 15:04:18.439390 27291 leader_election.cc:288] T 00000000000000000000000000000000 P 16f7d668b467495b83b117535850e625 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 15:04:18.439421 27294 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 16f7d668b467495b83b117535850e625 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 15:04:18.439558 27294 raft_consensus.cc:686] T 00000000000000000000000000000000 P 16f7d668b467495b83b117535850e625 [term 1 LEADER]: Becoming Leader. State: Replica: 16f7d668b467495b83b117535850e625, State: Running, Role: LEADER
I20260923 15:04:18.439677 27291 sys_catalog.cc:564] T 00000000000000000000000000000000 P 16f7d668b467495b83b117535850e625 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 15:04:18.439702 27294 consensus_queue.cc:237] T 00000000000000000000000000000000 P 16f7d668b467495b83b117535850e625 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16f7d668b467495b83b117535850e625" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 46395 } }
I20260923 15:04:18.440069 27296 sys_catalog.cc:455] T 00000000000000000000000000000000 P 16f7d668b467495b83b117535850e625 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 16f7d668b467495b83b117535850e625. Latest consensus state: current_term: 1 leader_uuid: "16f7d668b467495b83b117535850e625" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16f7d668b467495b83b117535850e625" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 46395 } } }
I20260923 15:04:18.440172 27296 sys_catalog.cc:458] T 00000000000000000000000000000000 P 16f7d668b467495b83b117535850e625 [sys.catalog]: This master's current role is: LEADER
I20260923 15:04:18.440217 27295 sys_catalog.cc:455] T 00000000000000000000000000000000 P 16f7d668b467495b83b117535850e625 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "16f7d668b467495b83b117535850e625" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "16f7d668b467495b83b117535850e625" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 46395 } } }
I20260923 15:04:18.440387 27295 sys_catalog.cc:458] T 00000000000000000000000000000000 P 16f7d668b467495b83b117535850e625 [sys.catalog]: This master's current role is: LEADER
I20260923 15:04:18.440502 27301 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 15:04:18.441061 27301 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 15:04:18.441787 13614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKC4soy/build/release/bin/kudu as pid 27223
I20260923 15:04:18.441900 13614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.IPKICACertWebServerEndPoints.1790175691491353-13614-0/minicluster-data/master-0/wal/instance
I20260923 15:04:18.442109 13614 external_mini_cluster.cc:1428] Setting key 1c15e2f64485ea9729253b3a4811ec55
I20260923 15:04:18.443022 27301 catalog_manager.cc:1347] Generated new cluster ID: 3ae08a82e3e441029eab2bd32ae0f541
I20260923 15:04:18.443074 27301 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 23 15:04:18 dist-test-slave-cnb0 krb5kdc[27210](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175858, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.75.190@KRBTEST.COM
I20260923 15:04:18.454314 27314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:18.442589 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59910 (local address 127.13.75.190:46395)
0923 15:04:18.442750 (+ 161us) server_negotiation.cc:206] Beginning negotiation
0923 15:04:18.442759 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0923 15:04:18.442791 (+ 32us) server_negotiation.cc:407] Connection header received
0923 15:04:18.442935 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:04:18.442942 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:04:18.442999 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:04:18.443139 (+ 140us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:04:18.443954 (+ 815us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:04:18.445539 (+ 1585us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:04:18.446571 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:04:18.447325 (+ 754us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:04:18.447452 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:04:18.450362 (+ 2910us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:04:18.450366 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:04:18.450367 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:04:18.450378 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:04:18.452424 (+ 2046us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:04:18.452895 (+ 471us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:04:18.452898 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:04:18.452899 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:04:18.453006 (+ 107us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:04:18.453242 (+ 236us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:04:18.453243 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:04:18.453243 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:04:18.453581 (+ 338us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:04:18.453724 (+ 143us) server_negotiation.cc:1035] Waiting for connection context
0923 15:04:18.453932 (+ 208us) server_negotiation.cc:299] Negotiation successful
0923 15:04:18.454072 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":112,"thread_start_us":67,"threads_started":1}
I20260923 15:04:18.455365 13614 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 15:04:18.466699 27301 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 15:04:18.467267 27301 catalog_manager.cc:1504] Loading token signing keys...
I20260923 15:04:18.470223 27301 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 16f7d668b467495b83b117535850e625: Generated new TSK 0
I20260923 15:04:18.470402 27301 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 15:04:18.509413 13614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKC4soy/build/release/bin/kudu with pid 27223
2026-09-23T15:04:18Z chronyd exiting
[ OK ] SecurityITest.IPKICACertWebServerEndPoints (1382 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedIPKICert
Loading random data
Initializing database '/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 15:04:18 dist-test-slave-cnb0 krb5kdc[27318](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 15:04:18 dist-test-slave-cnb0 krb5kdc[27318](info): set up 2 sockets
Sep 23 15:04:18 dist-test-slave-cnb0 krb5kdc[27318](info): commencing operation
krb5kdc: starting...
W20260923 15:04:19.552167 13614 mini_kdc.cc:121] Time spent starting KDC: real 1.025s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/krb5kdc/test-user.keytab.
Sep 23 15:04:19 dist-test-slave-cnb0 krb5kdc[27318](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175859, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T15:04:19Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T15:04:19Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.75.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.75.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.75.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.190.keytab.
Entry for principal kudu/127.13.75.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.190.keytab.
I20260923 15:04:19.674268 13614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKC4soy/build/release/bin/kudu
/tmp/dist-test-taskKC4soy/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.75.190:46259
--webserver_interface=127.13.75.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.190.keytab
--principal=kudu/127.13.75.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.75.148:44253
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.75.190:46259
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:04:19.756160 27331 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:04:19.756382 27331 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:04:19.756409 27331 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:04:19.757550 27331 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 15:04:19.757594 27331 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:04:19.757611 27331 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:04:19.757624 27331 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 15:04:19.757638 27331 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 15:04:19.759091 27331 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.75.148:44253
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.75.190:46259
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.190.keytab
--principal=kudu/127.13.75.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.75.190:46259
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.75.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27331
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:04:19.759303 27331 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:04:19.759490 27331 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:04:19.764405 27337 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:04:19.764480 27339 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:04:19.764487 27336 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:04:19.764596 27331 server_base.cc:1034] running on GCE node
I20260923 15:04:19.764900 27331 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:04:19.765241 27331 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:04:19.766388 27331 hybrid_clock.cc:648] HybridClock initialized: now 1790175859766370 us; error 36 us; skew 500 ppm
Sep 23 15:04:19 dist-test-slave-cnb0 krb5kdc[27318](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175859, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:04:19.768982 27331 init.cc:377] Logged in from keytab as kudu/127.13.75.190@KRBTEST.COM (short username kudu)
I20260923 15:04:19.769472 27331 webserver.cc:460] Webserver started at http://127.13.75.190:38245/ using document root <none> and password file <none>
I20260923 15:04:19.769711 27331 fs_manager.cc:359] Metadata directory not provided
I20260923 15:04:19.769778 27331 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:04:19.769930 27331 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:04:19.771067 27331 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/minicluster-data/master-0/data/instance:
uuid: "00499138dec94c0d888a87ea532d713f"
format_stamp: "Formatted at 2026-09-23 15:04:19 on dist-test-slave-cnb0"
server_key: "397714a7c6559323a9394756f5aabe94"
server_key_iv: "2213b3efe8a8ba55a3451100dc7ef6ea"
server_key_version: "encryptionkey@0"
I20260923 15:04:19.771435 27331 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/minicluster-data/master-0/wal/instance:
uuid: "00499138dec94c0d888a87ea532d713f"
format_stamp: "Formatted at 2026-09-23 15:04:19 on dist-test-slave-cnb0"
server_key: "397714a7c6559323a9394756f5aabe94"
server_key_iv: "2213b3efe8a8ba55a3451100dc7ef6ea"
server_key_version: "encryptionkey@0"
I20260923 15:04:19.772807 27331 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 15:04:19.773653 27346 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:04:19.773847 27331 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:04:19.773957 27331 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/minicluster-data/master-0/data,/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/minicluster-data/master-0/wal
uuid: "00499138dec94c0d888a87ea532d713f"
format_stamp: "Formatted at 2026-09-23 15:04:19 on dist-test-slave-cnb0"
server_key: "397714a7c6559323a9394756f5aabe94"
server_key_iv: "2213b3efe8a8ba55a3451100dc7ef6ea"
server_key_version: "encryptionkey@0"
I20260923 15:04:19.774031 27331 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:04:19.785838 27331 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:04:19.788259 27331 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:04:19.788434 27331 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:04:19.792620 27331 rpc_server.cc:307] RPC server started. Bound to: 127.13.75.190:46259
I20260923 15:04:19.792655 27398 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.75.190:46259 every 8 connection(s)
I20260923 15:04:19.793092 27331 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/minicluster-data/master-0/data/info.pb
I20260923 15:04:19.794083 27399 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:04:19.796474 27399 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 00499138dec94c0d888a87ea532d713f: Bootstrap starting.
I20260923 15:04:19.797062 27399 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 00499138dec94c0d888a87ea532d713f: Neither blocks nor log segments found. Creating new log.
I20260923 15:04:19.797341 27399 log.cc:824] T 00000000000000000000000000000000 P 00499138dec94c0d888a87ea532d713f: Log is configured to *not* fsync() on all Append() calls
I20260923 15:04:19.798069 27399 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 00499138dec94c0d888a87ea532d713f: No bootstrap required, opened a new log
I20260923 15:04:19.798977 13614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKC4soy/build/release/bin/kudu as pid 27331
I20260923 15:04:19.799082 13614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/minicluster-data/master-0/wal/instance
I20260923 15:04:19.799283 13614 external_mini_cluster.cc:1428] Setting key 135d3e8dec7fb90983136d7cdf8094be
I20260923 15:04:19.800288 27399 raft_consensus.cc:348] T 00000000000000000000000000000000 P 00499138dec94c0d888a87ea532d713f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00499138dec94c0d888a87ea532d713f" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 46259 } }
I20260923 15:04:19.800467 27399 raft_consensus.cc:374] T 00000000000000000000000000000000 P 00499138dec94c0d888a87ea532d713f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:04:19.800510 27399 raft_consensus.cc:729] T 00000000000000000000000000000000 P 00499138dec94c0d888a87ea532d713f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00499138dec94c0d888a87ea532d713f, State: Initialized, Role: FOLLOWER
I20260923 15:04:19.800648 27399 consensus_queue.cc:260] T 00000000000000000000000000000000 P 00499138dec94c0d888a87ea532d713f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00499138dec94c0d888a87ea532d713f" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 46259 } }
I20260923 15:04:19.800738 27399 raft_consensus.cc:388] T 00000000000000000000000000000000 P 00499138dec94c0d888a87ea532d713f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 15:04:19.800779 27399 raft_consensus.cc:482] T 00000000000000000000000000000000 P 00499138dec94c0d888a87ea532d713f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 15:04:19.800834 27399 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 00499138dec94c0d888a87ea532d713f [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:04:19.801847 27399 raft_consensus.cc:504] T 00000000000000000000000000000000 P 00499138dec94c0d888a87ea532d713f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00499138dec94c0d888a87ea532d713f" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 46259 } }
I20260923 15:04:19.801980 27399 leader_election.cc:302] T 00000000000000000000000000000000 P 00499138dec94c0d888a87ea532d713f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 00499138dec94c0d888a87ea532d713f; no voters:
I20260923 15:04:19.802143 27399 leader_election.cc:288] T 00000000000000000000000000000000 P 00499138dec94c0d888a87ea532d713f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 15:04:19.802179 27404 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 00499138dec94c0d888a87ea532d713f [term 1 FOLLOWER]: Leader election won for term 1
I20260923 15:04:19.802464 27399 sys_catalog.cc:564] T 00000000000000000000000000000000 P 00499138dec94c0d888a87ea532d713f [sys.catalog]: configured and running, proceeding with master startup.
I20260923 15:04:19.802484 27404 raft_consensus.cc:686] T 00000000000000000000000000000000 P 00499138dec94c0d888a87ea532d713f [term 1 LEADER]: Becoming Leader. State: Replica: 00499138dec94c0d888a87ea532d713f, State: Running, Role: LEADER
I20260923 15:04:19.803124 27404 consensus_queue.cc:237] T 00000000000000000000000000000000 P 00499138dec94c0d888a87ea532d713f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00499138dec94c0d888a87ea532d713f" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 46259 } }
I20260923 15:04:19.804616 27405 sys_catalog.cc:455] T 00000000000000000000000000000000 P 00499138dec94c0d888a87ea532d713f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 00499138dec94c0d888a87ea532d713f. Latest consensus state: current_term: 1 leader_uuid: "00499138dec94c0d888a87ea532d713f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00499138dec94c0d888a87ea532d713f" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 46259 } } }
I20260923 15:04:19.804699 27405 sys_catalog.cc:458] T 00000000000000000000000000000000 P 00499138dec94c0d888a87ea532d713f [sys.catalog]: This master's current role is: LEADER
I20260923 15:04:19.804889 27406 sys_catalog.cc:455] T 00000000000000000000000000000000 P 00499138dec94c0d888a87ea532d713f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "00499138dec94c0d888a87ea532d713f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00499138dec94c0d888a87ea532d713f" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 46259 } } }
I20260923 15:04:19.805016 27406 sys_catalog.cc:458] T 00000000000000000000000000000000 P 00499138dec94c0d888a87ea532d713f [sys.catalog]: This master's current role is: LEADER
W20260923 15:04:19.805096 27419 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 00499138dec94c0d888a87ea532d713f: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260923 15:04:19.805138 27419 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260923 15:04:19.805205 27420 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 15:04:19.805410 27420 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Sep 23 15:04:19 dist-test-slave-cnb0 krb5kdc[27318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175859, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.75.190@KRBTEST.COM
I20260923 15:04:19.807021 27420 catalog_manager.cc:1347] Generated new cluster ID: 42cd13b126f348e88b37a4472b55607d
I20260923 15:04:19.807077 27420 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 15:04:19.810228 27403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:19.799829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34038 (local address 127.13.75.190:46259)
0923 15:04:19.800020 (+ 191us) server_negotiation.cc:206] Beginning negotiation
0923 15:04:19.800029 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0923 15:04:19.800063 (+ 34us) server_negotiation.cc:407] Connection header received
0923 15:04:19.800249 (+ 186us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:04:19.800254 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:04:19.800298 (+ 44us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:04:19.800403 (+ 105us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:04:19.801226 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:04:19.802227 (+ 1001us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:04:19.803318 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:04:19.803963 (+ 645us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:04:19.804086 (+ 123us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:04:19.806910 (+ 2824us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:04:19.806914 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:04:19.806914 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:04:19.806928 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:04:19.808553 (+ 1625us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:04:19.808941 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:04:19.808944 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:04:19.808945 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:04:19.809064 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:04:19.809305 (+ 241us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:04:19.809306 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:04:19.809306 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:04:19.809625 (+ 319us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:04:19.809765 (+ 140us) server_negotiation.cc:1035] Waiting for connection context
0923 15:04:19.809875 (+ 110us) server_negotiation.cc:299] Negotiation successful
0923 15:04:19.810027 (+ 152us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":78,"threads_started":1}
I20260923 15:04:19.812445 27420 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 15:04:19.812956 27420 catalog_manager.cc:1504] Loading token signing keys...
I20260923 15:04:19.816854 27420 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 00499138dec94c0d888a87ea532d713f: Generated new TSK 0
I20260923 15:04:19.817031 27420 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.75.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.75.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.75.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.129.keytab.
Entry for principal kudu/127.13.75.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.129.keytab.
I20260923 15:04:19.847332 13614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKC4soy/build/release/bin/kudu
/tmp/dist-test-taskKC4soy/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.75.129:0
--local_ip_for_outbound_sockets=127.13.75.129
--webserver_interface=127.13.75.129
--webserver_port=0
--tserver_master_addrs=127.13.75.190:46259
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.129.keytab
--principal=kudu/127.13.75.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.75.148:44253
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:04:19.929364 27425 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:04:19.929567 27425 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:04:19.929592 27425 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:04:19.930759 27425 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:04:19.930801 27425 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:04:19.930828 27425 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.75.129
I20260923 15:04:19.932452 27425 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.75.148:44253
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.129.keytab
--principal=kudu/127.13.75.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.75.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.75.129
--webserver_port=0
--tserver_master_addrs=127.13.75.190:46259
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27425
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.75.129
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:04:19.932696 27425 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:04:19.932942 27425 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:04:19.937945 27430 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:04:19.937986 27431 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:04:19.937990 27433 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:04:19.938150 27425 server_base.cc:1034] running on GCE node
I20260923 15:04:19.938403 27425 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:04:19.938648 27425 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:04:19.939852 27425 hybrid_clock.cc:648] HybridClock initialized: now 1790175859939826 us; error 35 us; skew 500 ppm
Sep 23 15:04:19 dist-test-slave-cnb0 krb5kdc[27318](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175859, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:04:19.942427 27425 init.cc:377] Logged in from keytab as kudu/127.13.75.129@KRBTEST.COM (short username kudu)
I20260923 15:04:19.942963 27425 webserver.cc:460] Webserver started at http://127.13.75.129:35765/ using document root <none> and password file <none>
I20260923 15:04:19.943186 27425 fs_manager.cc:359] Metadata directory not provided
I20260923 15:04:19.943252 27425 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:04:19.943403 27425 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:04:19.944478 27425 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/minicluster-data/ts-0/data/instance:
uuid: "f6496d7f1e484285ba207e9b3f30edd8"
format_stamp: "Formatted at 2026-09-23 15:04:19 on dist-test-slave-cnb0"
server_key: "a7714ab93270211d1d05b2a4cc05be9b"
server_key_iv: "edffafa014fff3f00c0a95d2207c6674"
server_key_version: "encryptionkey@0"
I20260923 15:04:19.944842 27425 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/minicluster-data/ts-0/wal/instance:
uuid: "f6496d7f1e484285ba207e9b3f30edd8"
format_stamp: "Formatted at 2026-09-23 15:04:19 on dist-test-slave-cnb0"
server_key: "a7714ab93270211d1d05b2a4cc05be9b"
server_key_iv: "edffafa014fff3f00c0a95d2207c6674"
server_key_version: "encryptionkey@0"
I20260923 15:04:19.946132 27425 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260923 15:04:19.946841 27440 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:04:19.947190 27425 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260923 15:04:19.947301 27425 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/minicluster-data/ts-0/wal
uuid: "f6496d7f1e484285ba207e9b3f30edd8"
format_stamp: "Formatted at 2026-09-23 15:04:19 on dist-test-slave-cnb0"
server_key: "a7714ab93270211d1d05b2a4cc05be9b"
server_key_iv: "edffafa014fff3f00c0a95d2207c6674"
server_key_version: "encryptionkey@0"
I20260923 15:04:19.947394 27425 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:04:19.962859 27425 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:04:19.965374 27425 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:04:19.965546 27425 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:04:19.965796 27425 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 15:04:19.966128 27425 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 15:04:19.966182 27425 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 15:04:19.966225 27425 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 15:04:19.966256 27425 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 15:04:19.972414 27425 rpc_server.cc:307] RPC server started. Bound to: 127.13.75.129:33885
I20260923 15:04:19.972460 27553 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.75.129:33885 every 8 connection(s)
I20260923 15:04:19.972787 27425 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/minicluster-data/ts-0/data/info.pb
Sep 23 15:04:19 dist-test-slave-cnb0 krb5kdc[27318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175859, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.129@KRBTEST.COM for kudu/127.13.75.190@KRBTEST.COM
I20260923 15:04:19.982542 13614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKC4soy/build/release/bin/kudu as pid 27425
I20260923 15:04:19.982650 27403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:19.973495 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.129:42315 (local address 127.13.75.190:46259)
0923 15:04:19.973541 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0923 15:04:19.973543 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:04:19.973740 (+ 197us) server_negotiation.cc:407] Connection header received
0923 15:04:19.974280 (+ 540us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:04:19.974282 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:04:19.974298 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:04:19.974359 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:04:19.975416 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:04:19.975940 (+ 524us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:04:19.977074 (+ 1134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:04:19.977306 (+ 232us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:04:19.977395 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:04:19.979758 (+ 2363us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:04:19.979760 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:04:19.979760 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:04:19.979771 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:04:19.981205 (+ 1434us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:04:19.981648 (+ 443us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:04:19.981649 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:04:19.981650 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:04:19.981708 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:04:19.981979 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:04:19.981980 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:04:19.981980 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:04:19.982098 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:04:19.982168 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0923 15:04:19.982462 (+ 294us) server_negotiation.cc:299] Negotiation successful
0923 15:04:19.982556 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260923 15:04:19.982661 13614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/minicluster-data/ts-0/wal/instance
I20260923 15:04:19.982868 13614 external_mini_cluster.cc:1428] Setting key 8d5b6093185a0b37372f988ee62f94b1
I20260923 15:04:19.982879 27556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:19.973461 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:46259 (local address 127.13.75.129:42315)
0923 15:04:19.973608 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0923 15:04:19.973654 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0923 15:04:19.974174 (+ 520us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:04:19.974430 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:04:19.974435 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:04:19.975098 (+ 663us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:04:19.975305 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:04:19.975310 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:04:19.976040 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:04:19.976041 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:04:19.976973 (+ 932us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:04:19.976976 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:04:19.977408 (+ 432us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:04:19.977409 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:04:19.977536 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:04:19.978145 (+ 609us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:04:19.978154 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:04:19.979652 (+ 1498us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:04:19.981306 (+ 1654us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:04:19.981314 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:04:19.981317 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:04:19.981555 (+ 238us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:04:19.981785 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:04:19.981786 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:04:19.981786 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:04:19.981902 (+ 116us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:04:19.982171 (+ 269us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:04:19.982179 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:04:19.982322 (+ 143us) client_negotiation.cc:769] Sending connection context
0923 15:04:19.982440 (+ 118us) client_negotiation.cc:240] Negotiation successful
0923 15:04:19.982608 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":104,"thread_start_us":70,"threads_started":1}
I20260923 15:04:19.983705 27554 heartbeater.cc:347] Connected to a master server at 127.13.75.190:46259
I20260923 15:04:19.983819 27554 heartbeater.cc:464] Registering TS with master...
I20260923 15:04:19.984006 27554 heartbeater.cc:511] Master 127.13.75.190:46259 requested a full tablet report, sending...
I20260923 15:04:19.984556 27363 ts_manager.cc:194] Registered new tserver with Master: f6496d7f1e484285ba207e9b3f30edd8 (127.13.75.129:33885)
I20260923 15:04:19.985502 27363 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.75.129@KRBTEST.COM'} at 127.13.75.129:42315
WARNING: no policy specified for kudu/127.13.75.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.75.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.75.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.130.keytab.
Entry for principal kudu/127.13.75.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.130.keytab.
I20260923 15:04:20.011620 13614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKC4soy/build/release/bin/kudu
/tmp/dist-test-taskKC4soy/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.75.130:0
--local_ip_for_outbound_sockets=127.13.75.130
--webserver_interface=127.13.75.130
--webserver_port=0
--tserver_master_addrs=127.13.75.190:46259
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.130.keytab
--principal=kudu/127.13.75.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.75.148:44253
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:04:20.095739 27559 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:04:20.095960 27559 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:04:20.095985 27559 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:04:20.097258 27559 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:04:20.097311 27559 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:04:20.097338 27559 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.75.130
I20260923 15:04:20.098775 27559 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.75.148:44253
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.130.keytab
--principal=kudu/127.13.75.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.75.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.75.130
--webserver_port=0
--tserver_master_addrs=127.13.75.190:46259
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27559
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.75.130
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:04:20.099059 27559 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:04:20.099306 27559 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:04:20.104430 27564 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:04:20.104494 27567 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:04:20.104552 27565 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:04:20.104794 27559 server_base.cc:1034] running on GCE node
I20260923 15:04:20.105093 27559 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:04:20.105423 27559 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:04:20.106621 27559 hybrid_clock.cc:648] HybridClock initialized: now 1790175860106605 us; error 25 us; skew 500 ppm
Sep 23 15:04:20 dist-test-slave-cnb0 krb5kdc[27318](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175860, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:04:20.109344 27559 init.cc:377] Logged in from keytab as kudu/127.13.75.130@KRBTEST.COM (short username kudu)
I20260923 15:04:20.109730 27559 webserver.cc:460] Webserver started at http://127.13.75.130:39943/ using document root <none> and password file <none>
I20260923 15:04:20.109920 27559 fs_manager.cc:359] Metadata directory not provided
I20260923 15:04:20.109959 27559 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:04:20.110049 27559 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:04:20.111157 27559 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/minicluster-data/ts-1/data/instance:
uuid: "5c449438d1ea448f8c0518eac20b8194"
format_stamp: "Formatted at 2026-09-23 15:04:20 on dist-test-slave-cnb0"
server_key: "0751eb9a74520531b5cb362090d40b54"
server_key_iv: "d8090d7eb65569f9b73c8389b6be7e1a"
server_key_version: "encryptionkey@0"
I20260923 15:04:20.111477 27559 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/minicluster-data/ts-1/wal/instance:
uuid: "5c449438d1ea448f8c0518eac20b8194"
format_stamp: "Formatted at 2026-09-23 15:04:20 on dist-test-slave-cnb0"
server_key: "0751eb9a74520531b5cb362090d40b54"
server_key_iv: "d8090d7eb65569f9b73c8389b6be7e1a"
server_key_version: "encryptionkey@0"
I20260923 15:04:20.112735 27559 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:04:20.113492 27574 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:04:20.113695 27559 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:04:20.113785 27559 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/minicluster-data/ts-1/wal
uuid: "5c449438d1ea448f8c0518eac20b8194"
format_stamp: "Formatted at 2026-09-23 15:04:20 on dist-test-slave-cnb0"
server_key: "0751eb9a74520531b5cb362090d40b54"
server_key_iv: "d8090d7eb65569f9b73c8389b6be7e1a"
server_key_version: "encryptionkey@0"
I20260923 15:04:20.113838 27559 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:04:20.127899 27559 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:04:20.130276 27559 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:04:20.130394 27559 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:04:20.130585 27559 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 15:04:20.130972 27559 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 15:04:20.131016 27559 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 15:04:20.131065 27559 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 15:04:20.131094 27559 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 15:04:20.137208 27559 rpc_server.cc:307] RPC server started. Bound to: 127.13.75.130:36107
I20260923 15:04:20.137282 27687 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.75.130:36107 every 8 connection(s)
I20260923 15:04:20.137641 27559 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/minicluster-data/ts-1/data/info.pb
Sep 23 15:04:20 dist-test-slave-cnb0 krb5kdc[27318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175860, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.130@KRBTEST.COM for kudu/127.13.75.190@KRBTEST.COM
I20260923 15:04:20.147032 13614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKC4soy/build/release/bin/kudu as pid 27559
I20260923 15:04:20.147145 13614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/minicluster-data/ts-1/wal/instance
I20260923 15:04:20.147362 13614 external_mini_cluster.cc:1428] Setting key 2d7bc1b05e782f1b9fe11c0abafe217e
I20260923 15:04:20.147971 27403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:20.138570 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.130:57159 (local address 127.13.75.190:46259)
0923 15:04:20.138625 (+ 55us) server_negotiation.cc:206] Beginning negotiation
0923 15:04:20.138627 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:04:20.138841 (+ 214us) server_negotiation.cc:407] Connection header received
0923 15:04:20.139453 (+ 612us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:04:20.139454 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:04:20.139466 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:04:20.139533 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:04:20.140659 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:04:20.141162 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:04:20.142228 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:04:20.142417 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:04:20.142510 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:04:20.144850 (+ 2340us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:04:20.144852 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:04:20.144852 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:04:20.144859 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:04:20.146300 (+ 1441us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:04:20.146810 (+ 510us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:04:20.146813 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:04:20.146814 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:04:20.146873 (+ 59us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:04:20.147204 (+ 331us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:04:20.147205 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:04:20.147205 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:04:20.147336 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:04:20.147429 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0923 15:04:20.147781 (+ 352us) server_negotiation.cc:299] Negotiation successful
0923 15:04:20.147881 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260923 15:04:20.148195 27690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:20.138499 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:46259 (local address 127.13.75.130:57159)
0923 15:04:20.138689 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0923 15:04:20.138738 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0923 15:04:20.139323 (+ 585us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:04:20.139656 (+ 333us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:04:20.139660 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:04:20.140327 (+ 667us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:04:20.140541 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:04:20.140547 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:04:20.141316 (+ 769us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:04:20.141317 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:04:20.142136 (+ 819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:04:20.142138 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:04:20.142540 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:04:20.142541 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:04:20.142651 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:04:20.143259 (+ 608us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:04:20.143267 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:04:20.144695 (+ 1428us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:04:20.146446 (+ 1751us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:04:20.146454 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:04:20.146456 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:04:20.146695 (+ 239us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:04:20.146984 (+ 289us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:04:20.146985 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:04:20.146986 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:04:20.147105 (+ 119us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:04:20.147423 (+ 318us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:04:20.147431 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:04:20.147637 (+ 206us) client_negotiation.cc:769] Sending connection context
0923 15:04:20.147746 (+ 109us) client_negotiation.cc:240] Negotiation successful
0923 15:04:20.147928 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":106,"threads_started":1}
I20260923 15:04:20.148696 27688 heartbeater.cc:347] Connected to a master server at 127.13.75.190:46259
I20260923 15:04:20.148837 27688 heartbeater.cc:464] Registering TS with master...
I20260923 15:04:20.149045 27688 heartbeater.cc:511] Master 127.13.75.190:46259 requested a full tablet report, sending...
I20260923 15:04:20.149430 27363 ts_manager.cc:194] Registered new tserver with Master: 5c449438d1ea448f8c0518eac20b8194 (127.13.75.130:36107)
I20260923 15:04:20.150014 27363 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.75.130@KRBTEST.COM'} at 127.13.75.130:57159
WARNING: no policy specified for kudu/127.13.75.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.75.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.75.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.131.keytab.
Entry for principal kudu/127.13.75.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.131.keytab.
I20260923 15:04:20.174639 13614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKC4soy/build/release/bin/kudu
/tmp/dist-test-taskKC4soy/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.75.131:0
--local_ip_for_outbound_sockets=127.13.75.131
--webserver_interface=127.13.75.131
--webserver_port=0
--tserver_master_addrs=127.13.75.190:46259
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.131.keytab
--principal=kudu/127.13.75.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.75.148:44253
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:04:20.258625 27693 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:04:20.258934 27693 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:04:20.258997 27693 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:04:20.260320 27693 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:04:20.260393 27693 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:04:20.260443 27693 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.75.131
I20260923 15:04:20.262123 27693 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.75.148:44253
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.131.keytab
--principal=kudu/127.13.75.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.75.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.75.131
--webserver_port=0
--tserver_master_addrs=127.13.75.190:46259
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27693
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.75.131
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:04:20.262432 27693 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:04:20.262693 27693 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:04:20.268225 27701 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:04:20.268290 27698 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:04:20.268464 27699 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:04:20.269028 27693 server_base.cc:1034] running on GCE node
I20260923 15:04:20.269266 27693 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:04:20.269495 27693 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:04:20.270649 27693 hybrid_clock.cc:648] HybridClock initialized: now 1790175860270636 us; error 29 us; skew 500 ppm
Sep 23 15:04:20 dist-test-slave-cnb0 krb5kdc[27318](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175860, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:04:20.273495 27693 init.cc:377] Logged in from keytab as kudu/127.13.75.131@KRBTEST.COM (short username kudu)
I20260923 15:04:20.273972 27693 webserver.cc:460] Webserver started at http://127.13.75.131:43299/ using document root <none> and password file <none>
I20260923 15:04:20.274228 27693 fs_manager.cc:359] Metadata directory not provided
I20260923 15:04:20.274298 27693 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:04:20.274446 27693 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:04:20.275727 27693 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/minicluster-data/ts-2/data/instance:
uuid: "53f9c19a5c7449faa9b3897191c6cfc6"
format_stamp: "Formatted at 2026-09-23 15:04:20 on dist-test-slave-cnb0"
server_key: "36bc0810866c35e78372225ca65d4ece"
server_key_iv: "3b934cb902d001ecaffb886f3a50f220"
server_key_version: "encryptionkey@0"
I20260923 15:04:20.276177 27693 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/minicluster-data/ts-2/wal/instance:
uuid: "53f9c19a5c7449faa9b3897191c6cfc6"
format_stamp: "Formatted at 2026-09-23 15:04:20 on dist-test-slave-cnb0"
server_key: "36bc0810866c35e78372225ca65d4ece"
server_key_iv: "3b934cb902d001ecaffb886f3a50f220"
server_key_version: "encryptionkey@0"
I20260923 15:04:20.278294 27693 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.002s sys 0.000s
I20260923 15:04:20.279189 27708 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:04:20.279414 27693 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:04:20.279525 27693 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/minicluster-data/ts-2/wal
uuid: "53f9c19a5c7449faa9b3897191c6cfc6"
format_stamp: "Formatted at 2026-09-23 15:04:20 on dist-test-slave-cnb0"
server_key: "36bc0810866c35e78372225ca65d4ece"
server_key_iv: "3b934cb902d001ecaffb886f3a50f220"
server_key_version: "encryptionkey@0"
I20260923 15:04:20.279623 27693 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:04:20.288507 27693 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:04:20.291101 27693 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:04:20.291400 27693 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:04:20.291819 27693 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 15:04:20.292186 27693 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 15:04:20.292223 27693 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 15:04:20.292268 27693 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 15:04:20.292294 27693 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 15:04:20.298151 27693 rpc_server.cc:307] RPC server started. Bound to: 127.13.75.131:40593
I20260923 15:04:20.298230 27821 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.75.131:40593 every 8 connection(s)
I20260923 15:04:20.298553 27693 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/minicluster-data/ts-2/data/info.pb
I20260923 15:04:20.299937 13614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKC4soy/build/release/bin/kudu as pid 27693
I20260923 15:04:20.300057 13614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/minicluster-data/ts-2/wal/instance
I20260923 15:04:20.300240 13614 external_mini_cluster.cc:1428] Setting key 1c96223aac461fcda95808768c7764e4
Sep 23 15:04:20 dist-test-slave-cnb0 krb5kdc[27318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175860, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.131@KRBTEST.COM for kudu/127.13.75.190@KRBTEST.COM
I20260923 15:04:20.308529 27403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:20.299432 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.131:50445 (local address 127.13.75.190:46259)
0923 15:04:20.299501 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0923 15:04:20.299504 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:04:20.299749 (+ 245us) server_negotiation.cc:407] Connection header received
0923 15:04:20.300316 (+ 567us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:04:20.300318 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:04:20.300334 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:04:20.300405 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:04:20.301607 (+ 1202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:04:20.302102 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:04:20.303449 (+ 1347us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:04:20.303638 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:04:20.303709 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:04:20.305898 (+ 2189us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:04:20.305899 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:04:20.305899 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:04:20.305905 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:04:20.307179 (+ 1274us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:04:20.307564 (+ 385us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:04:20.307565 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:04:20.307565 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:04:20.307616 (+ 51us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:04:20.307850 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:04:20.307852 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:04:20.307852 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:04:20.307970 (+ 118us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:04:20.308047 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0923 15:04:20.308397 (+ 350us) server_negotiation.cc:299] Negotiation successful
0923 15:04:20.308466 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260923 15:04:20.308772 27824 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:20.299431 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:46259 (local address 127.13.75.131:50445)
0923 15:04:20.299622 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0923 15:04:20.299659 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0923 15:04:20.300206 (+ 547us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:04:20.300506 (+ 300us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:04:20.300510 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:04:20.301243 (+ 733us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:04:20.301495 (+ 252us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:04:20.301500 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:04:20.302260 (+ 760us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:04:20.302261 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:04:20.303372 (+ 1111us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:04:20.303374 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:04:20.303726 (+ 352us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:04:20.303727 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:04:20.303839 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:04:20.304393 (+ 554us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:04:20.304401 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:04:20.305782 (+ 1381us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:04:20.307247 (+ 1465us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:04:20.307253 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:04:20.307255 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:04:20.307499 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:04:20.307675 (+ 176us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:04:20.307675 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:04:20.307676 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:04:20.307770 (+ 94us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:04:20.308071 (+ 301us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:04:20.308081 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:04:20.308206 (+ 125us) client_negotiation.cc:769] Sending connection context
0923 15:04:20.308362 (+ 156us) client_negotiation.cc:240] Negotiation successful
0923 15:04:20.308497 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":154,"thread_start_us":96,"threads_started":1}
I20260923 15:04:20.309286 27822 heartbeater.cc:347] Connected to a master server at 127.13.75.190:46259
I20260923 15:04:20.309379 27822 heartbeater.cc:464] Registering TS with master...
I20260923 15:04:20.309516 27822 heartbeater.cc:511] Master 127.13.75.190:46259 requested a full tablet report, sending...
I20260923 15:04:20.309893 27363 ts_manager.cc:194] Registered new tserver with Master: 53f9c19a5c7449faa9b3897191c6cfc6 (127.13.75.131:40593)
I20260923 15:04:20.310289 27363 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.75.131@KRBTEST.COM'} at 127.13.75.131:50445
I20260923 15:04:20.312194 13614 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260923 15:04:20.320410 27403 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:20.313531 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34052 (local address 127.13.75.190:46259)
0923 15:04:20.313619 (+ 88us) server_negotiation.cc:206] Beginning negotiation
0923 15:04:20.313621 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:04:20.313707 (+ 86us) server_negotiation.cc:407] Connection header received
0923 15:04:20.313807 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:04:20.313808 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:04:20.313819 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:04:20.313864 (+ 45us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:04:20.314493 (+ 629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:04:20.315055 (+ 562us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:04:20.315809 (+ 754us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:04:20.316023 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:04:20.316108 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:04:20.317644 (+ 1536us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:04:20.317645 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:04:20.317646 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:04:20.317654 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:04:20.319113 (+ 1459us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:04:20.319500 (+ 387us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:04:20.319502 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:04:20.319502 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:04:20.319568 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:04:20.319791 (+ 223us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:04:20.319792 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:04:20.319793 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:04:20.319970 (+ 177us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:04:20.320093 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0923 15:04:20.320209 (+ 116us) server_negotiation.cc:299] Negotiation successful
0923 15:04:20.320294 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":59}
I20260923 15:04:20.322257 27363 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34052:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 15:04:20.324141 27363 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 15:04:20.331557 27833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:20.326778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.131:40593 (local address 127.0.0.1:46520)
0923 15:04:20.327168 (+ 390us) negotiation.cc:107] Waiting for socket to connect
0923 15:04:20.327208 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0923 15:04:20.327278 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:04:20.327600 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:04:20.327603 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:04:20.327606 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:04:20.327752 (+ 146us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:04:20.327755 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:04:20.328881 (+ 1126us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:04:20.328883 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:04:20.330243 (+ 1360us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:04:20.330246 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:04:20.331198 (+ 952us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:04:20.331199 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:04:20.331302 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:04:20.331318 (+ 16us) client_negotiation.cc:769] Sending connection context
0923 15:04:20.331401 (+ 83us) client_negotiation.cc:240] Negotiation successful
0923 15:04:20.331454 (+ 53us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":366,"thread_start_us":57,"threads_started":1}
I20260923 15:04:20.331574 27835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:20.326944 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46520 (local address 127.13.75.131:40593)
0923 15:04:20.327403 (+ 459us) server_negotiation.cc:206] Beginning negotiation
0923 15:04:20.327407 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 15:04:20.327419 (+ 12us) server_negotiation.cc:407] Connection header received
0923 15:04:20.327465 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:04:20.327467 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:04:20.327486 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:04:20.327585 (+ 99us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:04:20.327829 (+ 244us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:04:20.328774 (+ 945us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:04:20.330341 (+ 1567us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:04:20.331108 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:04:20.331186 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:04:20.331244 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0923 15:04:20.331436 (+ 192us) server_negotiation.cc:299] Negotiation successful
0923 15:04:20.331483 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":421,"thread_start_us":117,"threads_started":1}
I20260923 15:04:20.331964 27837 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:20.327037 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.130:36107 (local address 127.0.0.1:37586)
0923 15:04:20.327408 (+ 371us) negotiation.cc:107] Waiting for socket to connect
0923 15:04:20.327417 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 15:04:20.327470 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:04:20.327920 (+ 450us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:04:20.327921 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:04:20.327924 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:04:20.327966 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:04:20.327968 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:04:20.329247 (+ 1279us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:04:20.329248 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:04:20.330478 (+ 1230us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:04:20.330480 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:04:20.331677 (+ 1197us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:04:20.331678 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:04:20.331738 (+ 60us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:04:20.331743 (+ 5us) client_negotiation.cc:769] Sending connection context
0923 15:04:20.331800 (+ 57us) client_negotiation.cc:240] Negotiation successful
0923 15:04:20.331849 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":352,"thread_start_us":73,"threads_started":1}
I20260923 15:04:20.332196 27838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:20.327168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37586 (local address 127.13.75.130:36107)
0923 15:04:20.327332 (+ 164us) server_negotiation.cc:206] Beginning negotiation
0923 15:04:20.327338 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 15:04:20.327465 (+ 127us) server_negotiation.cc:407] Connection header received
0923 15:04:20.327722 (+ 257us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:04:20.327725 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:04:20.327744 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:04:20.327830 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:04:20.328040 (+ 210us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:04:20.329131 (+ 1091us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:04:20.330562 (+ 1431us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:04:20.331342 (+ 780us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:04:20.331435 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:04:20.331496 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0923 15:04:20.332070 (+ 574us) server_negotiation.cc:299] Negotiation successful
0923 15:04:20.332115 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":127,"thread_start_us":105,"threads_started":1}
I20260923 15:04:20.332474 27756 tablet_service.cc:1467] Processing CreateTablet for tablet 6355f7c0a83c436384785489a220e688 (DEFAULT_TABLE table=test-table [id=047a47c0ce8e4c9e90ee79a753608a6e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 15:04:20.332908 27834 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:20.326777 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.129:33885 (local address 127.0.0.1:51268)
0923 15:04:20.327165 (+ 388us) negotiation.cc:107] Waiting for socket to connect
0923 15:04:20.327208 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0923 15:04:20.327294 (+ 86us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:04:20.327832 (+ 538us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:04:20.327833 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:04:20.327835 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:04:20.327866 (+ 31us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:04:20.327879 (+ 13us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:04:20.330554 (+ 2675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:04:20.330555 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:04:20.331508 (+ 953us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:04:20.331511 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:04:20.332638 (+ 1127us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:04:20.332640 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:04:20.332712 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:04:20.332717 (+ 5us) client_negotiation.cc:769] Sending connection context
0923 15:04:20.332782 (+ 65us) client_negotiation.cc:240] Negotiation successful
0923 15:04:20.332829 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":350,"thread_start_us":126,"threads_started":1}
I20260923 15:04:20.332895 27756 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6355f7c0a83c436384785489a220e688. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:04:20.333195 27836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:20.326940 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51268 (local address 127.13.75.129:33885)
0923 15:04:20.327607 (+ 667us) server_negotiation.cc:206] Beginning negotiation
0923 15:04:20.327613 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 15:04:20.327622 (+ 9us) server_negotiation.cc:407] Connection header received
0923 15:04:20.327657 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:04:20.327660 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:04:20.327676 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:04:20.327751 (+ 75us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:04:20.327951 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:04:20.329130 (+ 1179us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:04:20.331579 (+ 2449us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:04:20.332551 (+ 972us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:04:20.332612 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:04:20.332648 (+ 36us) server_negotiation.cc:1035] Waiting for connection context
0923 15:04:20.333052 (+ 404us) server_negotiation.cc:299] Negotiation successful
0923 15:04:20.333106 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":623,"thread_start_us":102,"threads_started":1}
I20260923 15:04:20.334015 27488 tablet_service.cc:1467] Processing CreateTablet for tablet 6355f7c0a83c436384785489a220e688 (DEFAULT_TABLE table=test-table [id=047a47c0ce8e4c9e90ee79a753608a6e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 15:04:20.334399 27488 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6355f7c0a83c436384785489a220e688. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:04:20.336778 27839 tablet_bootstrap.cc:492] T 6355f7c0a83c436384785489a220e688 P 53f9c19a5c7449faa9b3897191c6cfc6: Bootstrap starting.
I20260923 15:04:20.337448 27839 tablet_bootstrap.cc:654] T 6355f7c0a83c436384785489a220e688 P 53f9c19a5c7449faa9b3897191c6cfc6: Neither blocks nor log segments found. Creating new log.
I20260923 15:04:20.337666 27622 tablet_service.cc:1467] Processing CreateTablet for tablet 6355f7c0a83c436384785489a220e688 (DEFAULT_TABLE table=test-table [id=047a47c0ce8e4c9e90ee79a753608a6e]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 15:04:20.337919 27840 tablet_bootstrap.cc:492] T 6355f7c0a83c436384785489a220e688 P f6496d7f1e484285ba207e9b3f30edd8: Bootstrap starting.
I20260923 15:04:20.338050 27622 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 6355f7c0a83c436384785489a220e688. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:04:20.338088 27839 log.cc:824] T 6355f7c0a83c436384785489a220e688 P 53f9c19a5c7449faa9b3897191c6cfc6: Log is configured to *not* fsync() on all Append() calls
I20260923 15:04:20.338626 27840 tablet_bootstrap.cc:654] T 6355f7c0a83c436384785489a220e688 P f6496d7f1e484285ba207e9b3f30edd8: Neither blocks nor log segments found. Creating new log.
I20260923 15:04:20.338985 27839 tablet_bootstrap.cc:492] T 6355f7c0a83c436384785489a220e688 P 53f9c19a5c7449faa9b3897191c6cfc6: No bootstrap required, opened a new log
I20260923 15:04:20.339361 27839 ts_tablet_manager.cc:1397] T 6355f7c0a83c436384785489a220e688 P 53f9c19a5c7449faa9b3897191c6cfc6: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260923 15:04:20.339444 27840 log.cc:824] T 6355f7c0a83c436384785489a220e688 P f6496d7f1e484285ba207e9b3f30edd8: Log is configured to *not* fsync() on all Append() calls
I20260923 15:04:20.340196 27840 tablet_bootstrap.cc:492] T 6355f7c0a83c436384785489a220e688 P f6496d7f1e484285ba207e9b3f30edd8: No bootstrap required, opened a new log
I20260923 15:04:20.340279 27840 ts_tablet_manager.cc:1397] T 6355f7c0a83c436384785489a220e688 P f6496d7f1e484285ba207e9b3f30edd8: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260923 15:04:20.341538 27843 tablet_bootstrap.cc:492] T 6355f7c0a83c436384785489a220e688 P 5c449438d1ea448f8c0518eac20b8194: Bootstrap starting.
I20260923 15:04:20.341449 27839 raft_consensus.cc:348] T 6355f7c0a83c436384785489a220e688 P 53f9c19a5c7449faa9b3897191c6cfc6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53f9c19a5c7449faa9b3897191c6cfc6" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 40593 } } peers { permanent_uuid: "5c449438d1ea448f8c0518eac20b8194" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 36107 } } peers { permanent_uuid: "f6496d7f1e484285ba207e9b3f30edd8" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 33885 } }
I20260923 15:04:20.341610 27839 raft_consensus.cc:374] T 6355f7c0a83c436384785489a220e688 P 53f9c19a5c7449faa9b3897191c6cfc6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:04:20.341652 27839 raft_consensus.cc:729] T 6355f7c0a83c436384785489a220e688 P 53f9c19a5c7449faa9b3897191c6cfc6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53f9c19a5c7449faa9b3897191c6cfc6, State: Initialized, Role: FOLLOWER
I20260923 15:04:20.341813 27839 consensus_queue.cc:260] T 6355f7c0a83c436384785489a220e688 P 53f9c19a5c7449faa9b3897191c6cfc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53f9c19a5c7449faa9b3897191c6cfc6" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 40593 } } peers { permanent_uuid: "5c449438d1ea448f8c0518eac20b8194" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 36107 } } peers { permanent_uuid: "f6496d7f1e484285ba207e9b3f30edd8" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 33885 } }
I20260923 15:04:20.342108 27839 ts_tablet_manager.cc:1428] T 6355f7c0a83c436384785489a220e688 P 53f9c19a5c7449faa9b3897191c6cfc6: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260923 15:04:20.341980 27840 raft_consensus.cc:348] T 6355f7c0a83c436384785489a220e688 P f6496d7f1e484285ba207e9b3f30edd8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53f9c19a5c7449faa9b3897191c6cfc6" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 40593 } } peers { permanent_uuid: "5c449438d1ea448f8c0518eac20b8194" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 36107 } } peers { permanent_uuid: "f6496d7f1e484285ba207e9b3f30edd8" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 33885 } }
I20260923 15:04:20.342149 27840 raft_consensus.cc:374] T 6355f7c0a83c436384785489a220e688 P f6496d7f1e484285ba207e9b3f30edd8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:04:20.342195 27840 raft_consensus.cc:729] T 6355f7c0a83c436384785489a220e688 P f6496d7f1e484285ba207e9b3f30edd8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6496d7f1e484285ba207e9b3f30edd8, State: Initialized, Role: FOLLOWER
I20260923 15:04:20.342238 27822 heartbeater.cc:503] Master 127.13.75.190:46259 was elected leader, sending a full tablet report...
I20260923 15:04:20.342310 27840 consensus_queue.cc:260] T 6355f7c0a83c436384785489a220e688 P f6496d7f1e484285ba207e9b3f30edd8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53f9c19a5c7449faa9b3897191c6cfc6" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 40593 } } peers { permanent_uuid: "5c449438d1ea448f8c0518eac20b8194" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 36107 } } peers { permanent_uuid: "f6496d7f1e484285ba207e9b3f30edd8" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 33885 } }
I20260923 15:04:20.342581 27843 tablet_bootstrap.cc:654] T 6355f7c0a83c436384785489a220e688 P 5c449438d1ea448f8c0518eac20b8194: Neither blocks nor log segments found. Creating new log.
I20260923 15:04:20.342594 27840 ts_tablet_manager.cc:1428] T 6355f7c0a83c436384785489a220e688 P f6496d7f1e484285ba207e9b3f30edd8: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260923 15:04:20.342993 27554 heartbeater.cc:503] Master 127.13.75.190:46259 was elected leader, sending a full tablet report...
I20260923 15:04:20.343282 27843 log.cc:824] T 6355f7c0a83c436384785489a220e688 P 5c449438d1ea448f8c0518eac20b8194: Log is configured to *not* fsync() on all Append() calls
I20260923 15:04:20.344199 27843 tablet_bootstrap.cc:492] T 6355f7c0a83c436384785489a220e688 P 5c449438d1ea448f8c0518eac20b8194: No bootstrap required, opened a new log
I20260923 15:04:20.344283 27843 ts_tablet_manager.cc:1397] T 6355f7c0a83c436384785489a220e688 P 5c449438d1ea448f8c0518eac20b8194: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260923 15:04:20.345638 27843 raft_consensus.cc:348] T 6355f7c0a83c436384785489a220e688 P 5c449438d1ea448f8c0518eac20b8194 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53f9c19a5c7449faa9b3897191c6cfc6" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 40593 } } peers { permanent_uuid: "5c449438d1ea448f8c0518eac20b8194" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 36107 } } peers { permanent_uuid: "f6496d7f1e484285ba207e9b3f30edd8" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 33885 } }
I20260923 15:04:20.345782 27843 raft_consensus.cc:374] T 6355f7c0a83c436384785489a220e688 P 5c449438d1ea448f8c0518eac20b8194 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:04:20.345826 27843 raft_consensus.cc:729] T 6355f7c0a83c436384785489a220e688 P 5c449438d1ea448f8c0518eac20b8194 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c449438d1ea448f8c0518eac20b8194, State: Initialized, Role: FOLLOWER
I20260923 15:04:20.345945 27843 consensus_queue.cc:260] T 6355f7c0a83c436384785489a220e688 P 5c449438d1ea448f8c0518eac20b8194 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53f9c19a5c7449faa9b3897191c6cfc6" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 40593 } } peers { permanent_uuid: "5c449438d1ea448f8c0518eac20b8194" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 36107 } } peers { permanent_uuid: "f6496d7f1e484285ba207e9b3f30edd8" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 33885 } }
I20260923 15:04:20.346213 27843 ts_tablet_manager.cc:1428] T 6355f7c0a83c436384785489a220e688 P 5c449438d1ea448f8c0518eac20b8194: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260923 15:04:20.346275 27688 heartbeater.cc:503] Master 127.13.75.190:46259 was elected leader, sending a full tablet report...
I20260923 15:04:20.378943 27845 raft_consensus.cc:482] T 6355f7c0a83c436384785489a220e688 P f6496d7f1e484285ba207e9b3f30edd8 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 15:04:20.379074 27845 raft_consensus.cc:504] T 6355f7c0a83c436384785489a220e688 P f6496d7f1e484285ba207e9b3f30edd8 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53f9c19a5c7449faa9b3897191c6cfc6" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 40593 } } peers { permanent_uuid: "5c449438d1ea448f8c0518eac20b8194" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 36107 } } peers { permanent_uuid: "f6496d7f1e484285ba207e9b3f30edd8" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 33885 } }
I20260923 15:04:20.379520 27845 leader_election.cc:288] T 6355f7c0a83c436384785489a220e688 P f6496d7f1e484285ba207e9b3f30edd8 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 53f9c19a5c7449faa9b3897191c6cfc6 (127.13.75.131:40593), 5c449438d1ea448f8c0518eac20b8194 (127.13.75.130:36107)
I20260923 15:04:20.382457 27556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:20.379691 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.131:40593 (local address 127.13.75.129:35897)
0923 15:04:20.379786 (+ 95us) negotiation.cc:107] Waiting for socket to connect
0923 15:04:20.379818 (+ 32us) client_negotiation.cc:174] Beginning negotiation
0923 15:04:20.379879 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:04:20.380037 (+ 158us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:04:20.380038 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:04:20.380040 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:04:20.380116 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:04:20.380118 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:04:20.380780 (+ 662us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:04:20.380781 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:04:20.381521 (+ 740us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:04:20.381523 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:04:20.382174 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:04:20.382175 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:04:20.382260 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:04:20.382263 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 15:04:20.382339 (+ 76us) client_negotiation.cc:240] Negotiation successful
0923 15:04:20.382382 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":61}
I20260923 15:04:20.382457 27835 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:20.379825 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.129:35897 (local address 127.13.75.131:40593)
0923 15:04:20.379884 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0923 15:04:20.379886 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:04:20.379942 (+ 56us) server_negotiation.cc:407] Connection header received
0923 15:04:20.379971 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:04:20.379972 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:04:20.379982 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:04:20.380032 (+ 50us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:04:20.380196 (+ 164us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:04:20.380686 (+ 490us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:04:20.381615 (+ 929us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:04:20.382092 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:04:20.382176 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:04:20.382186 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0923 15:04:20.382348 (+ 162us) server_negotiation.cc:299] Negotiation successful
0923 15:04:20.382384 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260923 15:04:20.382696 27848 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:20.379818 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.130:36107 (local address 127.13.75.129:34109)
0923 15:04:20.380007 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0923 15:04:20.380014 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0923 15:04:20.380083 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:04:20.380229 (+ 146us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:04:20.380230 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:04:20.380231 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:04:20.380306 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:04:20.380308 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:04:20.380949 (+ 641us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:04:20.380951 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:04:20.381679 (+ 728us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:04:20.381681 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:04:20.382390 (+ 709us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:04:20.382391 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:04:20.382472 (+ 81us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:04:20.382475 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 15:04:20.382541 (+ 66us) client_negotiation.cc:240] Negotiation successful
0923 15:04:20.382592 (+ 51us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":113,"threads_started":1}
I20260923 15:04:20.382773 27776 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6355f7c0a83c436384785489a220e688" candidate_uuid: "f6496d7f1e484285ba207e9b3f30edd8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53f9c19a5c7449faa9b3897191c6cfc6" is_pre_election: true
I20260923 15:04:20.382849 27838 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:20.379889 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.129:34109 (local address 127.13.75.130:36107)
0923 15:04:20.379954 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0923 15:04:20.379955 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 15:04:20.380073 (+ 118us) server_negotiation.cc:407] Connection header received
0923 15:04:20.380147 (+ 74us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:04:20.380148 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:04:20.380157 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:04:20.380224 (+ 67us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:04:20.380379 (+ 155us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:04:20.380860 (+ 481us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:04:20.381777 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:04:20.382308 (+ 531us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:04:20.382390 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:04:20.382398 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0923 15:04:20.382668 (+ 270us) server_negotiation.cc:299] Negotiation successful
0923 15:04:20.382709 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260923 15:04:20.382994 27776 raft_consensus.cc:2454] T 6355f7c0a83c436384785489a220e688 P 53f9c19a5c7449faa9b3897191c6cfc6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f6496d7f1e484285ba207e9b3f30edd8 in term 0.
I20260923 15:04:20.383142 27642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6355f7c0a83c436384785489a220e688" candidate_uuid: "f6496d7f1e484285ba207e9b3f30edd8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c449438d1ea448f8c0518eac20b8194" is_pre_election: true
I20260923 15:04:20.383235 27441 leader_election.cc:302] T 6355f7c0a83c436384785489a220e688 P f6496d7f1e484285ba207e9b3f30edd8 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53f9c19a5c7449faa9b3897191c6cfc6, f6496d7f1e484285ba207e9b3f30edd8; no voters:
I20260923 15:04:20.383315 27642 raft_consensus.cc:2454] T 6355f7c0a83c436384785489a220e688 P 5c449438d1ea448f8c0518eac20b8194 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate f6496d7f1e484285ba207e9b3f30edd8 in term 0.
I20260923 15:04:20.383350 27845 raft_consensus.cc:2781] T 6355f7c0a83c436384785489a220e688 P f6496d7f1e484285ba207e9b3f30edd8 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 15:04:20.383405 27845 raft_consensus.cc:482] T 6355f7c0a83c436384785489a220e688 P f6496d7f1e484285ba207e9b3f30edd8 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 15:04:20.383440 27845 raft_consensus.cc:3037] T 6355f7c0a83c436384785489a220e688 P f6496d7f1e484285ba207e9b3f30edd8 [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:04:20.384167 27845 raft_consensus.cc:504] T 6355f7c0a83c436384785489a220e688 P f6496d7f1e484285ba207e9b3f30edd8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53f9c19a5c7449faa9b3897191c6cfc6" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 40593 } } peers { permanent_uuid: "5c449438d1ea448f8c0518eac20b8194" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 36107 } } peers { permanent_uuid: "f6496d7f1e484285ba207e9b3f30edd8" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 33885 } }
I20260923 15:04:20.384298 27845 leader_election.cc:288] T 6355f7c0a83c436384785489a220e688 P f6496d7f1e484285ba207e9b3f30edd8 [CANDIDATE]: Term 1 election: Requested vote from peers 53f9c19a5c7449faa9b3897191c6cfc6 (127.13.75.131:40593), 5c449438d1ea448f8c0518eac20b8194 (127.13.75.130:36107)
I20260923 15:04:20.384495 27776 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6355f7c0a83c436384785489a220e688" candidate_uuid: "f6496d7f1e484285ba207e9b3f30edd8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53f9c19a5c7449faa9b3897191c6cfc6"
I20260923 15:04:20.384490 27642 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "6355f7c0a83c436384785489a220e688" candidate_uuid: "f6496d7f1e484285ba207e9b3f30edd8" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "5c449438d1ea448f8c0518eac20b8194"
I20260923 15:04:20.384565 27642 raft_consensus.cc:3037] T 6355f7c0a83c436384785489a220e688 P 5c449438d1ea448f8c0518eac20b8194 [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:04:20.384581 27776 raft_consensus.cc:3037] T 6355f7c0a83c436384785489a220e688 P 53f9c19a5c7449faa9b3897191c6cfc6 [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:04:20.385257 27776 raft_consensus.cc:2454] T 6355f7c0a83c436384785489a220e688 P 53f9c19a5c7449faa9b3897191c6cfc6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f6496d7f1e484285ba207e9b3f30edd8 in term 1.
I20260923 15:04:20.385293 27642 raft_consensus.cc:2454] T 6355f7c0a83c436384785489a220e688 P 5c449438d1ea448f8c0518eac20b8194 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate f6496d7f1e484285ba207e9b3f30edd8 in term 1.
I20260923 15:04:20.385587 27441 leader_election.cc:302] T 6355f7c0a83c436384785489a220e688 P f6496d7f1e484285ba207e9b3f30edd8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c449438d1ea448f8c0518eac20b8194, f6496d7f1e484285ba207e9b3f30edd8; no voters:
I20260923 15:04:20.385902 27845 raft_consensus.cc:2781] T 6355f7c0a83c436384785489a220e688 P f6496d7f1e484285ba207e9b3f30edd8 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 15:04:20.386099 27845 raft_consensus.cc:686] T 6355f7c0a83c436384785489a220e688 P f6496d7f1e484285ba207e9b3f30edd8 [term 1 LEADER]: Becoming Leader. State: Replica: f6496d7f1e484285ba207e9b3f30edd8, State: Running, Role: LEADER
I20260923 15:04:20.386205 27845 consensus_queue.cc:237] T 6355f7c0a83c436384785489a220e688 P f6496d7f1e484285ba207e9b3f30edd8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53f9c19a5c7449faa9b3897191c6cfc6" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 40593 } } peers { permanent_uuid: "5c449438d1ea448f8c0518eac20b8194" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 36107 } } peers { permanent_uuid: "f6496d7f1e484285ba207e9b3f30edd8" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 33885 } }
I20260923 15:04:20.387068 27363 catalog_manager.cc:5495] T 6355f7c0a83c436384785489a220e688 P f6496d7f1e484285ba207e9b3f30edd8 reported cstate change: term changed from 0 to 1, leader changed from <none> to f6496d7f1e484285ba207e9b3f30edd8 (127.13.75.129). New cstate: current_term: 1 leader_uuid: "f6496d7f1e484285ba207e9b3f30edd8" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53f9c19a5c7449faa9b3897191c6cfc6" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 40593 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5c449438d1ea448f8c0518eac20b8194" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 36107 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f6496d7f1e484285ba207e9b3f30edd8" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 33885 } health_report { overall_health: HEALTHY } } }
W20260923 15:04:20.388181 27689 tablet.cc:2369] T 6355f7c0a83c436384785489a220e688 P 5c449438d1ea448f8c0518eac20b8194: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 15:04:20.405352 27836 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:20.402744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51276 (local address 127.13.75.129:33885)
0923 15:04:20.402805 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0923 15:04:20.402807 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:04:20.402836 (+ 29us) server_negotiation.cc:407] Connection header received
0923 15:04:20.402916 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:04:20.402918 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:04:20.402938 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:04:20.403023 (+ 85us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 15:04:20.403162 (+ 139us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:04:20.403719 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:04:20.404464 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:04:20.404746 (+ 282us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:04:20.404817 (+ 71us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:04:20.404995 (+ 178us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 15:04:20.405108 (+ 113us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 15:04:20.405203 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0923 15:04:20.405222 (+ 19us) server_negotiation.cc:299] Negotiation successful
0923 15:04:20.405243 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260923 15:04:20.407191 27776 raft_consensus.cc:1261] T 6355f7c0a83c436384785489a220e688 P 53f9c19a5c7449faa9b3897191c6cfc6 [term 1 FOLLOWER]: Refusing update from remote peer f6496d7f1e484285ba207e9b3f30edd8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 15:04:20.407230 27642 raft_consensus.cc:1261] T 6355f7c0a83c436384785489a220e688 P 5c449438d1ea448f8c0518eac20b8194 [term 1 FOLLOWER]: Refusing update from remote peer f6496d7f1e484285ba207e9b3f30edd8: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 15:04:20.407572 27845 consensus_queue.cc:1035] T 6355f7c0a83c436384785489a220e688 P f6496d7f1e484285ba207e9b3f30edd8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53f9c19a5c7449faa9b3897191c6cfc6" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 40593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 15:04:20.407727 27849 consensus_queue.cc:1035] T 6355f7c0a83c436384785489a220e688 P f6496d7f1e484285ba207e9b3f30edd8 [LEADER]: Connected to new peer: Peer: permanent_uuid: "5c449438d1ea448f8c0518eac20b8194" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 36107 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 15:04:20.411021 27854 mvcc.cc:205] Tried to move back new op lower bound from 7332560324224901120 to 7332560324142514176. Current Snapshot: MvccSnapshot[applied={T|T < 7332560324224901120}]
I20260923 15:04:20.412539 27362 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34052:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 15:04:20.412633 27362 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34052:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 15:04:20.413411 27362 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 00499138dec94c0d888a87ea532d713f: Sending DeleteTablet for 3 replicas of tablet 6355f7c0a83c436384785489a220e688
I20260923 15:04:20.413765 13614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKC4soy/build/release/bin/kudu with pid 27425
I20260923 15:04:20.413753 27756 tablet_service.cc:1514] Processing DeleteTablet for tablet 6355f7c0a83c436384785489a220e688 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 15:04:20 UTC) from {username='kudu', principal='kudu/127.13.75.190@KRBTEST.COM'} at 127.0.0.1:46520
I20260923 15:04:20.413789 27622 tablet_service.cc:1514] Processing DeleteTablet for tablet 6355f7c0a83c436384785489a220e688 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 15:04:20 UTC) from {username='kudu', principal='kudu/127.13.75.190@KRBTEST.COM'} at 127.0.0.1:37586
W20260923 15:04:20.418852 27348 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.75.129:33885 (error 108)
I20260923 15:04:20.418915 27862 tablet_replica.cc:330] T 6355f7c0a83c436384785489a220e688 P 5c449438d1ea448f8c0518eac20b8194: stopping tablet replica
I20260923 15:04:20.418964 27863 tablet_replica.cc:330] T 6355f7c0a83c436384785489a220e688 P 53f9c19a5c7449faa9b3897191c6cfc6: stopping tablet replica
I20260923 15:04:20.419039 27862 raft_consensus.cc:2229] T 6355f7c0a83c436384785489a220e688 P 5c449438d1ea448f8c0518eac20b8194 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 15:04:20.419059 27863 raft_consensus.cc:2229] T 6355f7c0a83c436384785489a220e688 P 53f9c19a5c7449faa9b3897191c6cfc6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 15:04:20.419155 27862 raft_consensus.cc:2258] T 6355f7c0a83c436384785489a220e688 P 5c449438d1ea448f8c0518eac20b8194 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 15:04:20.419171 27863 raft_consensus.cc:2258] T 6355f7c0a83c436384785489a220e688 P 53f9c19a5c7449faa9b3897191c6cfc6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 15:04:20.419204 13614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKC4soy/build/release/bin/kudu with pid 27559
I20260923 15:04:20.420323 27863 ts_tablet_manager.cc:1905] T 6355f7c0a83c436384785489a220e688 P 53f9c19a5c7449faa9b3897191c6cfc6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 15:04:20.421373 27833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:20.421235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.129:33885 (local address 127.0.0.1:51286)
0923 15:04:20.421298 (+ 63us) negotiation.cc:107] Waiting for socket to connect
0923 15:04:20.421343 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.129:33885: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260923 15:04:20.421505 27348 catalog_manager.cc:4584] TS f6496d7f1e484285ba207e9b3f30edd8 (127.13.75.129:33885): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 6355f7c0a83c436384785489a220e688: Network error: Client connection negotiation failed: client connection to 127.13.75.129:33885: connect: Connection refused (error 111)
I20260923 15:04:20.422348 27863 ts_tablet_manager.cc:1918] T 6355f7c0a83c436384785489a220e688 P 53f9c19a5c7449faa9b3897191c6cfc6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 15:04:20.422436 27863 log.cc:1196] T 6355f7c0a83c436384785489a220e688 P 53f9c19a5c7449faa9b3897191c6cfc6: Deleting WAL directory at /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/minicluster-data/ts-2/wal/wals/6355f7c0a83c436384785489a220e688
I20260923 15:04:20.422698 27863 ts_tablet_manager.cc:1939] T 6355f7c0a83c436384785489a220e688 P 53f9c19a5c7449faa9b3897191c6cfc6: Deleting consensus metadata
I20260923 15:04:20.423349 27347 catalog_manager.cc:4841] TS 53f9c19a5c7449faa9b3897191c6cfc6 (127.13.75.131:40593): tablet 6355f7c0a83c436384785489a220e688 (table test-table [id=047a47c0ce8e4c9e90ee79a753608a6e]) successfully deleted
I20260923 15:04:20.424634 13614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKC4soy/build/release/bin/kudu with pid 27693
I20260923 15:04:20.424650 27833 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:20.424519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.130:36107 (local address 127.0.0.1:37596)
0923 15:04:20.424588 (+ 69us) negotiation.cc:107] Waiting for socket to connect
0923 15:04:20.424620 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.130:36107: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":46}
I20260923 15:04:20.429363 13614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKC4soy/build/release/bin/kudu with pid 27331
I20260923 15:04:20.434367 13614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKC4soy/build/release/bin/kudu
/tmp/dist-test-taskKC4soy/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.75.190:46259
--webserver_interface=127.13.75.190
--webserver_port=38245
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.190.keytab
--principal=kudu/127.13.75.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.75.148:44253
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.75.190:46259
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--ipki_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:04:20.514379 27865 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:04:20.514583 27865 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:04:20.514605 27865 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:04:20.515762 27865 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 15:04:20.515803 27865 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:04:20.515817 27865 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:04:20.515830 27865 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 15:04:20.515842 27865 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 15:04:20.517323 27865 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.75.148:44253
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/minicluster-data/master-0/wal
--ipki_private_key_password_cmd=echo foo
--ipki_ca_key_size=768
--master_addresses=127.13.75.190:46259
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.190.keytab
--principal=kudu/127.13.75.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.75.190:46259
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.75.190
--webserver_port=38245
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27865
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:04:20.517576 27865 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:04:20.517805 27865 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:04:20.527661 27872 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:04:20.527661 27874 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:04:20.527755 27871 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:04:20.527979 27865 server_base.cc:1034] running on GCE node
I20260923 15:04:20.528235 27865 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:04:20.528625 27865 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:04:20.529768 27865 hybrid_clock.cc:648] HybridClock initialized: now 1790175860529747 us; error 35 us; skew 500 ppm
Sep 23 15:04:20 dist-test-slave-cnb0 krb5kdc[27318](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175860, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:04:20.532316 27865 init.cc:377] Logged in from keytab as kudu/127.13.75.190@KRBTEST.COM (short username kudu)
I20260923 15:04:20.532721 27865 webserver.cc:460] Webserver started at http://127.13.75.190:38245/ using document root <none> and password file <none>
I20260923 15:04:20.532968 27865 fs_manager.cc:359] Metadata directory not provided
I20260923 15:04:20.533059 27865 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:04:20.534475 27865 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:04:20.535343 27881 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:04:20.535676 27865 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260923 15:04:20.535792 27865 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/minicluster-data/master-0/data,/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/minicluster-data/master-0/wal
uuid: "00499138dec94c0d888a87ea532d713f"
format_stamp: "Formatted at 2026-09-23 15:04:19 on dist-test-slave-cnb0"
server_key: "397714a7c6559323a9394756f5aabe94"
server_key_iv: "2213b3efe8a8ba55a3451100dc7ef6ea"
server_key_version: "encryptionkey@0"
I20260923 15:04:20.536248 27865 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:04:20.555558 27865 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:04:20.557590 27865 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:04:20.557772 27865 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:04:20.562320 27865 rpc_server.cc:307] RPC server started. Bound to: 127.13.75.190:46259
I20260923 15:04:20.562354 27934 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.75.190:46259 every 8 connection(s)
I20260923 15:04:20.562803 27865 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/minicluster-data/master-0/data/info.pb
I20260923 15:04:20.564739 27935 sys_catalog.cc:263] Verifying existing consensus state
I20260923 15:04:20.565496 27935 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 00499138dec94c0d888a87ea532d713f: Bootstrap starting.
I20260923 15:04:20.569229 13614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKC4soy/build/release/bin/kudu as pid 27865
I20260923 15:04:20.569563 13614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKC4soy/build/release/bin/kudu
/tmp/dist-test-taskKC4soy/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.75.129:33885
--local_ip_for_outbound_sockets=127.13.75.129
--tserver_master_addrs=127.13.75.190:46259
--webserver_port=35765
--webserver_interface=127.13.75.129
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.129.keytab
--principal=kudu/127.13.75.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.75.148:44253
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260923 15:04:20.575352 27935 log.cc:824] T 00000000000000000000000000000000 P 00499138dec94c0d888a87ea532d713f: Log is configured to *not* fsync() on all Append() calls
I20260923 15:04:20.577438 27935 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 00499138dec94c0d888a87ea532d713f: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260923 15:04:20.577800 27935 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 00499138dec94c0d888a87ea532d713f: Bootstrap complete.
I20260923 15:04:20.580492 27935 raft_consensus.cc:348] T 00000000000000000000000000000000 P 00499138dec94c0d888a87ea532d713f [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00499138dec94c0d888a87ea532d713f" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 46259 } }
I20260923 15:04:20.580739 27935 raft_consensus.cc:729] T 00000000000000000000000000000000 P 00499138dec94c0d888a87ea532d713f [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00499138dec94c0d888a87ea532d713f, State: Initialized, Role: FOLLOWER
I20260923 15:04:20.580940 27935 consensus_queue.cc:260] T 00000000000000000000000000000000 P 00499138dec94c0d888a87ea532d713f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00499138dec94c0d888a87ea532d713f" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 46259 } }
I20260923 15:04:20.581008 27935 raft_consensus.cc:388] T 00000000000000000000000000000000 P 00499138dec94c0d888a87ea532d713f [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 15:04:20.581077 27935 raft_consensus.cc:482] T 00000000000000000000000000000000 P 00499138dec94c0d888a87ea532d713f [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 15:04:20.581156 27935 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 00499138dec94c0d888a87ea532d713f [term 1 FOLLOWER]: Advancing to term 2
I20260923 15:04:20.582319 27935 raft_consensus.cc:504] T 00000000000000000000000000000000 P 00499138dec94c0d888a87ea532d713f [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00499138dec94c0d888a87ea532d713f" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 46259 } }
I20260923 15:04:20.582581 27935 leader_election.cc:302] T 00000000000000000000000000000000 P 00499138dec94c0d888a87ea532d713f [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 00499138dec94c0d888a87ea532d713f; no voters:
I20260923 15:04:20.582861 27939 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 00499138dec94c0d888a87ea532d713f [term 2 FOLLOWER]: Leader election won for term 2
I20260923 15:04:20.582844 27935 leader_election.cc:288] T 00000000000000000000000000000000 P 00499138dec94c0d888a87ea532d713f [CANDIDATE]: Term 2 election: Requested vote from peers
I20260923 15:04:20.583047 27939 raft_consensus.cc:686] T 00000000000000000000000000000000 P 00499138dec94c0d888a87ea532d713f [term 2 LEADER]: Becoming Leader. State: Replica: 00499138dec94c0d888a87ea532d713f, State: Running, Role: LEADER
I20260923 15:04:20.583151 27939 consensus_queue.cc:237] T 00000000000000000000000000000000 P 00499138dec94c0d888a87ea532d713f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00499138dec94c0d888a87ea532d713f" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 46259 } }
I20260923 15:04:20.583220 27935 sys_catalog.cc:564] T 00000000000000000000000000000000 P 00499138dec94c0d888a87ea532d713f [sys.catalog]: configured and running, proceeding with master startup.
I20260923 15:04:20.583446 27940 sys_catalog.cc:455] T 00000000000000000000000000000000 P 00499138dec94c0d888a87ea532d713f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 00499138dec94c0d888a87ea532d713f. Latest consensus state: current_term: 2 leader_uuid: "00499138dec94c0d888a87ea532d713f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00499138dec94c0d888a87ea532d713f" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 46259 } } }
I20260923 15:04:20.583515 27939 sys_catalog.cc:455] T 00000000000000000000000000000000 P 00499138dec94c0d888a87ea532d713f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "00499138dec94c0d888a87ea532d713f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00499138dec94c0d888a87ea532d713f" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 46259 } } }
I20260923 15:04:20.583628 27939 sys_catalog.cc:458] T 00000000000000000000000000000000 P 00499138dec94c0d888a87ea532d713f [sys.catalog]: This master's current role is: LEADER
I20260923 15:04:20.583609 27940 sys_catalog.cc:458] T 00000000000000000000000000000000 P 00499138dec94c0d888a87ea532d713f [sys.catalog]: This master's current role is: LEADER
I20260923 15:04:20.584044 27946 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 15:04:20.584887 27946 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 15:04:20.585168 27946 catalog_manager.cc:1259] Loaded cluster ID: 42cd13b126f348e88b37a4472b55607d
I20260923 15:04:20.585325 27946 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 15:04:20.585619 27946 catalog_manager.cc:1314] IPKI private key was stored in cleartext, attempting to encrypt it.
I20260923 15:04:20.590687 27946 catalog_manager.cc:1324] IPKI private key successfully encrypted.
I20260923 15:04:20.593176 27946 catalog_manager.cc:1504] Loading token signing keys...
I20260923 15:04:20.593365 27946 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 00499138dec94c0d888a87ea532d713f: Loaded TSK: 0
I20260923 15:04:20.593575 27946 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260923 15:04:20.657312 27937 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:04:20.657523 27937 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:04:20.657547 27937 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:04:20.658735 27937 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:04:20.658776 27937 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:04:20.658802 27937 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.75.129
I20260923 15:04:20.660386 27937 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.75.148:44253
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.129.keytab
--principal=kudu/127.13.75.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.75.129:33885
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.75.129
--webserver_port=35765
--tserver_master_addrs=127.13.75.190:46259
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.27937
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.75.129
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:04:20.660636 27937 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:04:20.660877 27937 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:04:20.666460 27962 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:04:20.666669 27937 server_base.cc:1034] running on GCE node
W20260923 15:04:20.666473 27963 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:04:20.666481 27965 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:04:20.667088 27937 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:04:20.667340 27937 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:04:20.668504 27937 hybrid_clock.cc:648] HybridClock initialized: now 1790175860668491 us; error 31 us; skew 500 ppm
Sep 23 15:04:20 dist-test-slave-cnb0 krb5kdc[27318](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175860, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:04:20.671159 27937 init.cc:377] Logged in from keytab as kudu/127.13.75.129@KRBTEST.COM (short username kudu)
I20260923 15:04:20.671638 27937 webserver.cc:460] Webserver started at http://127.13.75.129:35765/ using document root <none> and password file <none>
I20260923 15:04:20.671871 27937 fs_manager.cc:359] Metadata directory not provided
I20260923 15:04:20.671957 27937 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:04:20.673359 27937 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260923 15:04:20.674113 27972 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:04:20.674268 27937 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260923 15:04:20.674345 27937 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/minicluster-data/ts-0/wal
uuid: "f6496d7f1e484285ba207e9b3f30edd8"
format_stamp: "Formatted at 2026-09-23 15:04:19 on dist-test-slave-cnb0"
server_key: "a7714ab93270211d1d05b2a4cc05be9b"
server_key_iv: "edffafa014fff3f00c0a95d2207c6674"
server_key_version: "encryptionkey@0"
I20260923 15:04:20.674680 27937 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:04:20.686192 27937 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:04:20.688403 27937 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:04:20.688563 27937 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:04:20.688797 27937 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 15:04:20.689239 27979 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260923 15:04:20.691214 27937 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260923 15:04:20.691296 27937 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260923 15:04:20.691344 27937 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260923 15:04:20.691980 27937 ts_tablet_manager.cc:610] Registered 1 tablets
I20260923 15:04:20.692037 27937 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260923 15:04:20.692112 27979 tablet_bootstrap.cc:492] T 6355f7c0a83c436384785489a220e688 P f6496d7f1e484285ba207e9b3f30edd8: Bootstrap starting.
I20260923 15:04:20.699280 27937 rpc_server.cc:307] RPC server started. Bound to: 127.13.75.129:33885
I20260923 15:04:20.699401 28086 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.75.129:33885 every 8 connection(s)
I20260923 15:04:20.699704 27937 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/minicluster-data/ts-0/data/info.pb
I20260923 15:04:20.703223 27979 log.cc:824] T 6355f7c0a83c436384785489a220e688 P f6496d7f1e484285ba207e9b3f30edd8: Log is configured to *not* fsync() on all Append() calls
I20260923 15:04:20.705376 13614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKC4soy/build/release/bin/kudu as pid 27937
I20260923 15:04:20.705673 13614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKC4soy/build/release/bin/kudu
/tmp/dist-test-taskKC4soy/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.75.130:36107
--local_ip_for_outbound_sockets=127.13.75.130
--tserver_master_addrs=127.13.75.190:46259
--webserver_port=39943
--webserver_interface=127.13.75.130
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.130.keytab
--principal=kudu/127.13.75.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.75.148:44253
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260923 15:04:20.706276 27979 tablet_bootstrap.cc:492] T 6355f7c0a83c436384785489a220e688 P f6496d7f1e484285ba207e9b3f30edd8: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260923 15:04:20.706584 27979 tablet_bootstrap.cc:492] T 6355f7c0a83c436384785489a220e688 P f6496d7f1e484285ba207e9b3f30edd8: Bootstrap complete.
I20260923 15:04:20.707885 27979 ts_tablet_manager.cc:1397] T 6355f7c0a83c436384785489a220e688 P f6496d7f1e484285ba207e9b3f30edd8: Time spent bootstrapping tablet: real 0.016s user 0.007s sys 0.007s
I20260923 15:04:20.709225 27979 raft_consensus.cc:348] T 6355f7c0a83c436384785489a220e688 P f6496d7f1e484285ba207e9b3f30edd8 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53f9c19a5c7449faa9b3897191c6cfc6" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 40593 } } peers { permanent_uuid: "5c449438d1ea448f8c0518eac20b8194" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 36107 } } peers { permanent_uuid: "f6496d7f1e484285ba207e9b3f30edd8" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 33885 } }
I20260923 15:04:20.709527 27979 raft_consensus.cc:729] T 6355f7c0a83c436384785489a220e688 P f6496d7f1e484285ba207e9b3f30edd8 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6496d7f1e484285ba207e9b3f30edd8, State: Initialized, Role: FOLLOWER
I20260923 15:04:20.709694 27979 consensus_queue.cc:260] T 6355f7c0a83c436384785489a220e688 P f6496d7f1e484285ba207e9b3f30edd8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53f9c19a5c7449faa9b3897191c6cfc6" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 40593 } } peers { permanent_uuid: "5c449438d1ea448f8c0518eac20b8194" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 36107 } } peers { permanent_uuid: "f6496d7f1e484285ba207e9b3f30edd8" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 33885 } }
I20260923 15:04:20.709967 27979 ts_tablet_manager.cc:1428] T 6355f7c0a83c436384785489a220e688 P f6496d7f1e484285ba207e9b3f30edd8: Time spent starting tablet: real 0.002s user 0.001s sys 0.002s
Sep 23 15:04:20 dist-test-slave-cnb0 krb5kdc[27318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175860, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.129@KRBTEST.COM for kudu/127.13.75.190@KRBTEST.COM
I20260923 15:04:20.716506 28090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:20.700514 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.129:57793 (local address 127.13.75.190:46259)
0923 15:04:20.703065 (+ 2551us) server_negotiation.cc:206] Beginning negotiation
0923 15:04:20.703073 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0923 15:04:20.703107 (+ 34us) server_negotiation.cc:407] Connection header received
0923 15:04:20.703222 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:04:20.703229 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:04:20.703322 (+ 93us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:04:20.703482 (+ 160us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:04:20.704517 (+ 1035us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:04:20.706134 (+ 1617us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:04:20.708102 (+ 1968us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:04:20.708754 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:04:20.708900 (+ 146us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:04:20.712039 (+ 3139us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:04:20.712043 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:04:20.712043 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:04:20.712055 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:04:20.714284 (+ 2229us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:04:20.714915 (+ 631us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:04:20.714918 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:04:20.714919 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:04:20.715041 (+ 122us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:04:20.715382 (+ 341us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:04:20.715383 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:04:20.715383 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:04:20.715671 (+ 288us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:04:20.715763 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0923 15:04:20.716165 (+ 402us) server_negotiation.cc:299] Negotiation successful
0923 15:04:20.716303 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":2497,"thread_start_us":97,"threads_started":1}
I20260923 15:04:20.717114 28089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:20.700421 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:46259 (local address 127.13.75.129:57793)
0923 15:04:20.700596 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0923 15:04:20.700639 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0923 15:04:20.701085 (+ 446us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:04:20.703643 (+ 2558us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:04:20.703648 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:04:20.704159 (+ 511us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:04:20.704386 (+ 227us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:04:20.704392 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:04:20.706287 (+ 1895us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:04:20.706288 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:04:20.707926 (+ 1638us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:04:20.707930 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:04:20.708884 (+ 954us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:04:20.708885 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:04:20.709024 (+ 139us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:04:20.709675 (+ 651us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:04:20.709687 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:04:20.711836 (+ 2149us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:04:20.714389 (+ 2553us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:04:20.714398 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:04:20.714400 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:04:20.714789 (+ 389us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:04:20.715127 (+ 338us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:04:20.715129 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:04:20.715129 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:04:20.715295 (+ 166us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:04:20.715765 (+ 470us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:04:20.715774 (+ 9us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:04:20.715964 (+ 190us) client_negotiation.cc:769] Sending connection context
0923 15:04:20.716105 (+ 141us) client_negotiation.cc:240] Negotiation successful
0923 15:04:20.716318 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":102,"threads_started":1}
I20260923 15:04:20.717978 28087 heartbeater.cc:347] Connected to a master server at 127.13.75.190:46259
I20260923 15:04:20.718101 28087 heartbeater.cc:464] Registering TS with master...
I20260923 15:04:20.718338 28087 heartbeater.cc:511] Master 127.13.75.190:46259 requested a full tablet report, sending...
I20260923 15:04:20.719292 27899 ts_manager.cc:194] Registered new tserver with Master: f6496d7f1e484285ba207e9b3f30edd8 (127.13.75.129:33885)
I20260923 15:04:20.720497 27899 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.75.129@KRBTEST.COM'} at 127.13.75.129:57793
I20260923 15:04:20.721629 28087 heartbeater.cc:503] Master 127.13.75.190:46259 was elected leader, sending a full tablet report...
I20260923 15:04:20.727303 28094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:20.719944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.129:33885 (local address 127.0.0.1:51294)
0923 15:04:20.723037 (+ 3093us) negotiation.cc:107] Waiting for socket to connect
0923 15:04:20.723081 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0923 15:04:20.723154 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:04:20.723486 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:04:20.723489 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:04:20.723491 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:04:20.723566 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:04:20.723569 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:04:20.725090 (+ 1521us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:04:20.725091 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:04:20.725929 (+ 838us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:04:20.725932 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:04:20.727025 (+ 1093us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:04:20.727026 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:04:20.727106 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:04:20.727118 (+ 12us) client_negotiation.cc:769] Sending connection context
0923 15:04:20.727186 (+ 68us) client_negotiation.cc:240] Negotiation successful
0923 15:04:20.727238 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":3055,"thread_start_us":93,"threads_started":1}
I20260923 15:04:20.727403 28095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:20.720208 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51294 (local address 127.13.75.129:33885)
0923 15:04:20.723302 (+ 3094us) server_negotiation.cc:206] Beginning negotiation
0923 15:04:20.723307 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 15:04:20.723316 (+ 9us) server_negotiation.cc:407] Connection header received
0923 15:04:20.723358 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:04:20.723360 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:04:20.723377 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:04:20.723639 (+ 262us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:04:20.723688 (+ 49us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:04:20.724988 (+ 1300us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:04:20.726022 (+ 1034us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:04:20.726936 (+ 914us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:04:20.727023 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:04:20.727095 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0923 15:04:20.727229 (+ 134us) server_negotiation.cc:299] Negotiation successful
0923 15:04:20.727289 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3060,"thread_start_us":83,"threads_started":1}
I20260923 15:04:20.727954 28017 tablet_service.cc:1514] Processing DeleteTablet for tablet 6355f7c0a83c436384785489a220e688 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 15:04:20 UTC) from {username='kudu', principal='kudu/127.13.75.190@KRBTEST.COM'} at 127.0.0.1:51294
I20260923 15:04:20.728705 28096 tablet_replica.cc:330] stopping tablet replica
I20260923 15:04:20.728794 28096 raft_consensus.cc:2229] T 6355f7c0a83c436384785489a220e688 P f6496d7f1e484285ba207e9b3f30edd8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 15:04:20.728891 28096 raft_consensus.cc:2258] T 6355f7c0a83c436384785489a220e688 P f6496d7f1e484285ba207e9b3f30edd8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 15:04:20.729228 28096 ts_tablet_manager.cc:1905] T 6355f7c0a83c436384785489a220e688 P f6496d7f1e484285ba207e9b3f30edd8: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 15:04:20.731402 28096 ts_tablet_manager.cc:1918] T 6355f7c0a83c436384785489a220e688 P f6496d7f1e484285ba207e9b3f30edd8: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 15:04:20.731494 28096 log.cc:1196] T 6355f7c0a83c436384785489a220e688 P f6496d7f1e484285ba207e9b3f30edd8: Deleting WAL directory at /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/minicluster-data/ts-0/wal/wals/6355f7c0a83c436384785489a220e688
I20260923 15:04:20.731736 28096 ts_tablet_manager.cc:1939] T 6355f7c0a83c436384785489a220e688 P f6496d7f1e484285ba207e9b3f30edd8: Deleting consensus metadata
I20260923 15:04:20.732251 27883 catalog_manager.cc:4841] TS f6496d7f1e484285ba207e9b3f30edd8 (127.13.75.129:33885): tablet 6355f7c0a83c436384785489a220e688 (table test-table [id=047a47c0ce8e4c9e90ee79a753608a6e]) successfully deleted
W20260923 15:04:20.800758 28092 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:04:20.800997 28092 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:04:20.801038 28092 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:04:20.802209 28092 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:04:20.802281 28092 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:04:20.802327 28092 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.75.130
I20260923 15:04:20.803910 28092 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.75.148:44253
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.130.keytab
--principal=kudu/127.13.75.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.75.130:36107
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.75.130
--webserver_port=39943
--tserver_master_addrs=127.13.75.190:46259
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28092
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.75.130
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:04:20.804201 28092 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:04:20.804454 28092 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:04:20.809579 28102 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:04:20.809546 28104 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:04:20.809535 28101 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:04:20.810037 28092 server_base.cc:1034] running on GCE node
I20260923 15:04:20.810263 28092 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:04:20.810534 28092 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:04:20.811714 28092 hybrid_clock.cc:648] HybridClock initialized: now 1790175860811695 us; error 30 us; skew 500 ppm
Sep 23 15:04:20 dist-test-slave-cnb0 krb5kdc[27318](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175860, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:04:20.814555 28092 init.cc:377] Logged in from keytab as kudu/127.13.75.130@KRBTEST.COM (short username kudu)
I20260923 15:04:20.815116 28092 webserver.cc:460] Webserver started at http://127.13.75.130:39943/ using document root <none> and password file <none>
I20260923 15:04:20.815349 28092 fs_manager.cc:359] Metadata directory not provided
I20260923 15:04:20.815434 28092 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:04:20.816812 28092 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:04:20.817519 28111 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:04:20.817732 28092 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:04:20.817847 28092 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/minicluster-data/ts-1/wal
uuid: "5c449438d1ea448f8c0518eac20b8194"
format_stamp: "Formatted at 2026-09-23 15:04:20 on dist-test-slave-cnb0"
server_key: "0751eb9a74520531b5cb362090d40b54"
server_key_iv: "d8090d7eb65569f9b73c8389b6be7e1a"
server_key_version: "encryptionkey@0"
I20260923 15:04:20.818250 28092 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:04:20.843657 28092 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:04:20.846284 28092 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:04:20.846470 28092 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:04:20.846745 28092 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 15:04:20.847290 28118 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260923 15:04:20.848773 28092 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260923 15:04:20.848838 28092 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260923 15:04:20.848886 28092 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260923 15:04:20.849661 28092 ts_tablet_manager.cc:610] Registered 1 tablets
I20260923 15:04:20.849721 28092 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260923 15:04:20.849787 28118 tablet_bootstrap.cc:492] T 6355f7c0a83c436384785489a220e688 P 5c449438d1ea448f8c0518eac20b8194: Bootstrap starting.
I20260923 15:04:20.857601 28092 rpc_server.cc:307] RPC server started. Bound to: 127.13.75.130:36107
I20260923 15:04:20.857695 28225 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.75.130:36107 every 8 connection(s)
I20260923 15:04:20.858062 28092 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/minicluster-data/ts-1/data/info.pb
I20260923 15:04:20.858961 28118 log.cc:824] T 6355f7c0a83c436384785489a220e688 P 5c449438d1ea448f8c0518eac20b8194: Log is configured to *not* fsync() on all Append() calls
I20260923 15:04:20.860930 28118 tablet_bootstrap.cc:492] T 6355f7c0a83c436384785489a220e688 P 5c449438d1ea448f8c0518eac20b8194: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260923 15:04:20.861205 28118 tablet_bootstrap.cc:492] T 6355f7c0a83c436384785489a220e688 P 5c449438d1ea448f8c0518eac20b8194: Bootstrap complete.
I20260923 15:04:20.862131 28118 ts_tablet_manager.cc:1397] T 6355f7c0a83c436384785489a220e688 P 5c449438d1ea448f8c0518eac20b8194: Time spent bootstrapping tablet: real 0.012s user 0.002s sys 0.007s
I20260923 15:04:20.863176 28118 raft_consensus.cc:348] T 6355f7c0a83c436384785489a220e688 P 5c449438d1ea448f8c0518eac20b8194 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53f9c19a5c7449faa9b3897191c6cfc6" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 40593 } } peers { permanent_uuid: "5c449438d1ea448f8c0518eac20b8194" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 36107 } } peers { permanent_uuid: "f6496d7f1e484285ba207e9b3f30edd8" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 33885 } }
I20260923 15:04:20.863547 28118 raft_consensus.cc:729] T 6355f7c0a83c436384785489a220e688 P 5c449438d1ea448f8c0518eac20b8194 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c449438d1ea448f8c0518eac20b8194, State: Initialized, Role: FOLLOWER
I20260923 15:04:20.863664 13614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKC4soy/build/release/bin/kudu as pid 28092
I20260923 15:04:20.863714 28118 consensus_queue.cc:260] T 6355f7c0a83c436384785489a220e688 P 5c449438d1ea448f8c0518eac20b8194 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53f9c19a5c7449faa9b3897191c6cfc6" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 40593 } } peers { permanent_uuid: "5c449438d1ea448f8c0518eac20b8194" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 36107 } } peers { permanent_uuid: "f6496d7f1e484285ba207e9b3f30edd8" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 33885 } }
I20260923 15:04:20.863947 13614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKC4soy/build/release/bin/kudu
/tmp/dist-test-taskKC4soy/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.75.131:40593
--local_ip_for_outbound_sockets=127.13.75.131
--tserver_master_addrs=127.13.75.190:46259
--webserver_port=43299
--webserver_interface=127.13.75.131
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.131.keytab
--principal=kudu/127.13.75.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.75.148:44253
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260923 15:04:20.863999 28118 ts_tablet_manager.cc:1428] T 6355f7c0a83c436384785489a220e688 P 5c449438d1ea448f8c0518eac20b8194: Time spent starting tablet: real 0.002s user 0.004s sys 0.000s
Sep 23 15:04:20 dist-test-slave-cnb0 krb5kdc[27318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175860, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.130@KRBTEST.COM for kudu/127.13.75.190@KRBTEST.COM
I20260923 15:04:20.868934 28090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:20.858848 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.130:35961 (local address 127.13.75.190:46259)
0923 15:04:20.858923 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0923 15:04:20.858926 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:04:20.859186 (+ 260us) server_negotiation.cc:407] Connection header received
0923 15:04:20.859750 (+ 564us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:04:20.859752 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:04:20.859769 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:04:20.859839 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:04:20.860506 (+ 667us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:04:20.861097 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:04:20.862309 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:04:20.862607 (+ 298us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:04:20.862646 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:04:20.865722 (+ 3076us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:04:20.865724 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:04:20.865724 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:04:20.865731 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:04:20.867357 (+ 1626us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:04:20.867833 (+ 476us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:04:20.867835 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:04:20.867835 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:04:20.867904 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:04:20.868189 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:04:20.868190 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:04:20.868191 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:04:20.868331 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:04:20.868416 (+ 85us) server_negotiation.cc:1035] Waiting for connection context
0923 15:04:20.868718 (+ 302us) server_negotiation.cc:299] Negotiation successful
0923 15:04:20.868823 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260923 15:04:20.869107 28228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:20.858799 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:46259 (local address 127.13.75.130:35961)
0923 15:04:20.859003 (+ 204us) negotiation.cc:107] Waiting for socket to connect
0923 15:04:20.859081 (+ 78us) client_negotiation.cc:174] Beginning negotiation
0923 15:04:20.859637 (+ 556us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:04:20.859904 (+ 267us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:04:20.859909 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:04:20.860191 (+ 282us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:04:20.860386 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:04:20.860391 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:04:20.861222 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:04:20.861223 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:04:20.862165 (+ 942us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:04:20.862190 (+ 25us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:04:20.862746 (+ 556us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:04:20.862747 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:04:20.862857 (+ 110us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:04:20.863531 (+ 674us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:04:20.863557 (+ 26us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:04:20.865610 (+ 2053us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:04:20.867474 (+ 1864us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:04:20.867482 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:04:20.867485 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:04:20.867728 (+ 243us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:04:20.867988 (+ 260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:04:20.867989 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:04:20.867990 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:04:20.868110 (+ 120us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:04:20.868405 (+ 295us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:04:20.868421 (+ 16us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:04:20.868569 (+ 148us) client_negotiation.cc:769] Sending connection context
0923 15:04:20.868693 (+ 124us) client_negotiation.cc:240] Negotiation successful
0923 15:04:20.868872 (+ 179us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":115,"threads_started":1}
I20260923 15:04:20.869796 28226 heartbeater.cc:347] Connected to a master server at 127.13.75.190:46259
I20260923 15:04:20.869911 28226 heartbeater.cc:464] Registering TS with master...
I20260923 15:04:20.870098 28226 heartbeater.cc:511] Master 127.13.75.190:46259 requested a full tablet report, sending...
I20260923 15:04:20.870718 27899 ts_manager.cc:194] Registered new tserver with Master: 5c449438d1ea448f8c0518eac20b8194 (127.13.75.130:36107)
I20260923 15:04:20.871425 27899 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.75.130@KRBTEST.COM'} at 127.13.75.130:35961
I20260923 15:04:20.872648 28226 heartbeater.cc:503] Master 127.13.75.190:46259 was elected leader, sending a full tablet report...
Sep 23 15:04:20 dist-test-slave-cnb0 krb5kdc[27318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175860, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.190@KRBTEST.COM for kudu/127.13.75.130@KRBTEST.COM
I20260923 15:04:20.880028 28094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:20.871092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.130:36107 (local address 127.0.0.1:37608)
0923 15:04:20.871561 (+ 469us) negotiation.cc:107] Waiting for socket to connect
0923 15:04:20.871573 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 15:04:20.871623 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:04:20.872094 (+ 471us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:04:20.872095 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:04:20.872400 (+ 305us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:04:20.872446 (+ 46us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:04:20.872449 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:04:20.873617 (+ 1168us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:04:20.873618 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:04:20.874164 (+ 546us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:04:20.874166 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:04:20.874690 (+ 524us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:04:20.874692 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:04:20.874770 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:04:20.875365 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:04:20.875370 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:04:20.876728 (+ 1358us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:04:20.878959 (+ 2231us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:04:20.878967 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:04:20.878968 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:04:20.879173 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:04:20.879401 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:04:20.879401 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:04:20.879402 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:04:20.879460 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:04:20.879774 (+ 314us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:04:20.879780 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:04:20.879825 (+ 45us) client_negotiation.cc:769] Sending connection context
0923 15:04:20.879879 (+ 54us) client_negotiation.cc:240] Negotiation successful
0923 15:04:20.879961 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":450}
I20260923 15:04:20.880131 28232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:20.871714 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37608 (local address 127.13.75.130:36107)
0923 15:04:20.871909 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0923 15:04:20.871928 (+ 19us) server_negotiation.cc:399] Waiting for connection header
0923 15:04:20.871945 (+ 17us) server_negotiation.cc:407] Connection header received
0923 15:04:20.871979 (+ 34us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:04:20.871981 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:04:20.871993 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:04:20.872074 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:04:20.872536 (+ 462us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:04:20.873529 (+ 993us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:04:20.874271 (+ 742us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:04:20.874604 (+ 333us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:04:20.874691 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:04:20.876819 (+ 2128us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:04:20.876827 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:04:20.876827 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:04:20.876834 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:04:20.878836 (+ 2002us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:04:20.879266 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:04:20.879270 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:04:20.879270 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:04:20.879332 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:04:20.879521 (+ 189us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:04:20.879522 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:04:20.879522 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:04:20.879694 (+ 172us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:04:20.879773 (+ 79us) server_negotiation.cc:1035] Waiting for connection context
0923 15:04:20.879937 (+ 164us) server_negotiation.cc:299] Negotiation successful
0923 15:04:20.880031 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":5,"server-negotiator.queue_time_us":144,"thread_start_us":101,"threads_started":1}
I20260923 15:04:20.880465 28160 tablet_service.cc:1514] Processing DeleteTablet for tablet 6355f7c0a83c436384785489a220e688 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 15:04:20 UTC) from {username='kudu', principal='kudu/127.13.75.190@KRBTEST.COM'} at 127.0.0.1:37608
I20260923 15:04:20.881018 28233 tablet_replica.cc:330] stopping tablet replica
I20260923 15:04:20.881146 28233 raft_consensus.cc:2229] T 6355f7c0a83c436384785489a220e688 P 5c449438d1ea448f8c0518eac20b8194 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 15:04:20.881230 28233 raft_consensus.cc:2258] T 6355f7c0a83c436384785489a220e688 P 5c449438d1ea448f8c0518eac20b8194 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 15:04:20.881642 28233 ts_tablet_manager.cc:1905] T 6355f7c0a83c436384785489a220e688 P 5c449438d1ea448f8c0518eac20b8194: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 15:04:20.883944 28233 ts_tablet_manager.cc:1918] T 6355f7c0a83c436384785489a220e688 P 5c449438d1ea448f8c0518eac20b8194: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 15:04:20.884037 28233 log.cc:1196] T 6355f7c0a83c436384785489a220e688 P 5c449438d1ea448f8c0518eac20b8194: Deleting WAL directory at /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/minicluster-data/ts-1/wal/wals/6355f7c0a83c436384785489a220e688
I20260923 15:04:20.884291 28233 ts_tablet_manager.cc:1939] T 6355f7c0a83c436384785489a220e688 P 5c449438d1ea448f8c0518eac20b8194: Deleting consensus metadata
I20260923 15:04:20.884855 27882 catalog_manager.cc:4841] TS 5c449438d1ea448f8c0518eac20b8194 (127.13.75.130:36107): tablet 6355f7c0a83c436384785489a220e688 (table test-table [id=047a47c0ce8e4c9e90ee79a753608a6e]) successfully deleted
W20260923 15:04:20.949941 28231 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:04:20.950165 28231 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:04:20.950202 28231 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:04:20.951417 28231 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:04:20.951478 28231 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:04:20.951524 28231 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.75.131
I20260923 15:04:20.952981 28231 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.75.148:44253
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.131.keytab
--principal=kudu/127.13.75.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.75.131:40593
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.75.131
--webserver_port=43299
--tserver_master_addrs=127.13.75.190:46259
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28231
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.75.131
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:04:20.953241 28231 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:04:20.953474 28231 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:04:20.958523 28239 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:04:20.958624 28241 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:04:20.958509 28238 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:04:20.958814 28231 server_base.cc:1034] running on GCE node
I20260923 15:04:20.959066 28231 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:04:20.959293 28231 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:04:20.960481 28231 hybrid_clock.cc:648] HybridClock initialized: now 1790175860960471 us; error 40 us; skew 500 ppm
Sep 23 15:04:20 dist-test-slave-cnb0 krb5kdc[27318](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175860, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:04:20.963003 28231 init.cc:377] Logged in from keytab as kudu/127.13.75.131@KRBTEST.COM (short username kudu)
I20260923 15:04:20.963542 28231 webserver.cc:460] Webserver started at http://127.13.75.131:43299/ using document root <none> and password file <none>
I20260923 15:04:20.963806 28231 fs_manager.cc:359] Metadata directory not provided
I20260923 15:04:20.963902 28231 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:04:20.965386 28231 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 15:04:20.966054 28248 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:04:20.966297 28231 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:04:20.966415 28231 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/minicluster-data/ts-2/wal
uuid: "53f9c19a5c7449faa9b3897191c6cfc6"
format_stamp: "Formatted at 2026-09-23 15:04:20 on dist-test-slave-cnb0"
server_key: "36bc0810866c35e78372225ca65d4ece"
server_key_iv: "3b934cb902d001ecaffb886f3a50f220"
server_key_version: "encryptionkey@0"
I20260923 15:04:20.966792 28231 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:04:20.980342 28231 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:04:20.982584 28231 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:04:20.982764 28231 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:04:20.983094 28231 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 15:04:20.983433 28231 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 15:04:20.983487 28231 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 15:04:20.983530 28231 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 15:04:20.983557 28231 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 15:04:20.989718 28231 rpc_server.cc:307] RPC server started. Bound to: 127.13.75.131:40593
I20260923 15:04:20.989763 28361 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.75.131:40593 every 8 connection(s)
I20260923 15:04:20.990106 28231 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/minicluster-data/ts-2/data/info.pb
Sep 23 15:04:20 dist-test-slave-cnb0 krb5kdc[27318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175860, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.131@KRBTEST.COM for kudu/127.13.75.190@KRBTEST.COM
I20260923 15:04:21.000015 13614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKC4soy/build/release/bin/kudu as pid 28231
I20260923 15:04:21.000463 28090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:20.990939 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.131:59279 (local address 127.13.75.190:46259)
0923 15:04:20.991015 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0923 15:04:20.991018 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:04:20.991370 (+ 352us) server_negotiation.cc:407] Connection header received
0923 15:04:20.991789 (+ 419us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:04:20.991791 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:04:20.991802 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:04:20.991860 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:04:20.992838 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:04:20.993435 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:04:20.994441 (+ 1006us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:04:20.994661 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:04:20.994727 (+ 66us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:04:20.997202 (+ 2475us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:04:20.997204 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:04:20.997204 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:04:20.997214 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:04:20.998706 (+ 1492us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:04:20.999280 (+ 574us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:04:20.999282 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:04:20.999282 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:04:20.999387 (+ 105us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:04:20.999682 (+ 295us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:04:20.999683 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:04:20.999683 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:04:20.999839 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:04:20.999915 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0923 15:04:21.000262 (+ 347us) server_negotiation.cc:299] Negotiation successful
0923 15:04:21.000372 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260923 15:04:21.000665 28364 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:20.990921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:46259 (local address 127.13.75.131:59279)
0923 15:04:20.991154 (+ 233us) negotiation.cc:107] Waiting for socket to connect
0923 15:04:20.991194 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0923 15:04:20.991697 (+ 503us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:04:20.991912 (+ 215us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:04:20.991915 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:04:20.992523 (+ 608us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:04:20.992716 (+ 193us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:04:20.992720 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:04:20.993527 (+ 807us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:04:20.993528 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:04:20.994354 (+ 826us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:04:20.994356 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:04:20.994758 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:04:20.994759 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:04:20.994870 (+ 111us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:04:20.995493 (+ 623us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:04:20.995503 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:04:20.997090 (+ 1587us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:04:20.998854 (+ 1764us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:04:20.998864 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:04:20.998867 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:04:20.999183 (+ 316us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:04:20.999466 (+ 283us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:04:20.999467 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:04:20.999467 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:04:20.999598 (+ 131us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:04:20.999917 (+ 319us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:04:20.999925 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:04:21.000104 (+ 179us) client_negotiation.cc:769] Sending connection context
0923 15:04:21.000211 (+ 107us) client_negotiation.cc:240] Negotiation successful
0923 15:04:21.000395 (+ 184us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":188,"thread_start_us":125,"threads_started":1}
I20260923 15:04:21.001206 28362 heartbeater.cc:347] Connected to a master server at 127.13.75.190:46259
I20260923 15:04:21.001329 28362 heartbeater.cc:464] Registering TS with master...
I20260923 15:04:21.001555 28362 heartbeater.cc:511] Master 127.13.75.190:46259 requested a full tablet report, sending...
I20260923 15:04:21.002050 27899 ts_manager.cc:194] Registered new tserver with Master: 53f9c19a5c7449faa9b3897191c6cfc6 (127.13.75.131:40593)
I20260923 15:04:21.002494 27899 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.75.131@KRBTEST.COM'} at 127.13.75.131:59279
I20260923 15:04:21.008042 28366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:21.000527 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34060 (local address 127.13.75.190:46259)
0923 15:04:21.000652 (+ 125us) server_negotiation.cc:206] Beginning negotiation
0923 15:04:21.000654 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:04:21.000906 (+ 252us) server_negotiation.cc:407] Connection header received
0923 15:04:21.000927 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:04:21.000928 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:04:21.000942 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:04:21.001003 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:04:21.001766 (+ 763us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:04:21.002392 (+ 626us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:04:21.003434 (+ 1042us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:04:21.003654 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:04:21.003721 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:04:21.005165 (+ 1444us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:04:21.005175 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:04:21.005175 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:04:21.005182 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:04:21.006777 (+ 1595us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:04:21.007172 (+ 395us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:04:21.007173 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:04:21.007173 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:04:21.007235 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:04:21.007520 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:04:21.007521 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:04:21.007522 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:04:21.007644 (+ 122us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:04:21.007771 (+ 127us) server_negotiation.cc:1035] Waiting for connection context
0923 15:04:21.007879 (+ 108us) server_negotiation.cc:299] Negotiation successful
0923 15:04:21.007967 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":103,"thread_start_us":75,"threads_started":1}
I20260923 15:04:21.009034 13614 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260923 15:04:21.012291 28366 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:21.009418 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34066 (local address 127.13.75.190:46259)
0923 15:04:21.009465 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0923 15:04:21.009467 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:04:21.009588 (+ 121us) server_negotiation.cc:407] Connection header received
0923 15:04:21.009683 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:04:21.009684 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:04:21.009730 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:04:21.009805 (+ 75us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 15:04:21.009962 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:04:21.010651 (+ 689us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:04:21.011488 (+ 837us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:04:21.011700 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:04:21.011776 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:04:21.011940 (+ 164us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 15:04:21.012058 (+ 118us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 15:04:21.012169 (+ 111us) server_negotiation.cc:1035] Waiting for connection context
0923 15:04:21.012185 (+ 16us) server_negotiation.cc:299] Negotiation successful
0923 15:04:21.012212 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260923 15:04:21.012691 27899 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin'} at 127.0.0.1:34066:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 15:04:21.014389 27899 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 15:04:21.016701 28017 tablet_service.cc:1467] Processing CreateTablet for tablet 4db10a4e0b6a404f94257b46eb1ac368 (DEFAULT_TABLE table=test-table [id=09b096ac9704473386a300178f335c0d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 15:04:21.016800 28160 tablet_service.cc:1467] Processing CreateTablet for tablet 4db10a4e0b6a404f94257b46eb1ac368 (DEFAULT_TABLE table=test-table [id=09b096ac9704473386a300178f335c0d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 15:04:21.016922 28017 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4db10a4e0b6a404f94257b46eb1ac368. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:04:21.017005 28160 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4db10a4e0b6a404f94257b46eb1ac368. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:04:21.018355 28118 tablet_bootstrap.cc:492] T 4db10a4e0b6a404f94257b46eb1ac368 P 5c449438d1ea448f8c0518eac20b8194: Bootstrap starting.
I20260923 15:04:21.018429 27979 tablet_bootstrap.cc:492] T 4db10a4e0b6a404f94257b46eb1ac368 P f6496d7f1e484285ba207e9b3f30edd8: Bootstrap starting.
I20260923 15:04:21.018688 28118 tablet_bootstrap.cc:654] T 4db10a4e0b6a404f94257b46eb1ac368 P 5c449438d1ea448f8c0518eac20b8194: Neither blocks nor log segments found. Creating new log.
I20260923 15:04:21.018817 27979 tablet_bootstrap.cc:654] T 4db10a4e0b6a404f94257b46eb1ac368 P f6496d7f1e484285ba207e9b3f30edd8: Neither blocks nor log segments found. Creating new log.
I20260923 15:04:21.019483 27979 tablet_bootstrap.cc:492] T 4db10a4e0b6a404f94257b46eb1ac368 P f6496d7f1e484285ba207e9b3f30edd8: No bootstrap required, opened a new log
I20260923 15:04:21.019539 27979 ts_tablet_manager.cc:1397] T 4db10a4e0b6a404f94257b46eb1ac368 P f6496d7f1e484285ba207e9b3f30edd8: Time spent bootstrapping tablet: real 0.001s user 0.000s sys 0.001s
I20260923 15:04:21.019737 27979 raft_consensus.cc:348] T 4db10a4e0b6a404f94257b46eb1ac368 P f6496d7f1e484285ba207e9b3f30edd8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53f9c19a5c7449faa9b3897191c6cfc6" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 40593 } } peers { permanent_uuid: "f6496d7f1e484285ba207e9b3f30edd8" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 33885 } } peers { permanent_uuid: "5c449438d1ea448f8c0518eac20b8194" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 36107 } }
I20260923 15:04:21.019802 27979 raft_consensus.cc:374] T 4db10a4e0b6a404f94257b46eb1ac368 P f6496d7f1e484285ba207e9b3f30edd8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:04:21.019835 27979 raft_consensus.cc:729] T 4db10a4e0b6a404f94257b46eb1ac368 P f6496d7f1e484285ba207e9b3f30edd8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6496d7f1e484285ba207e9b3f30edd8, State: Initialized, Role: FOLLOWER
I20260923 15:04:21.019912 27979 consensus_queue.cc:260] T 4db10a4e0b6a404f94257b46eb1ac368 P f6496d7f1e484285ba207e9b3f30edd8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53f9c19a5c7449faa9b3897191c6cfc6" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 40593 } } peers { permanent_uuid: "f6496d7f1e484285ba207e9b3f30edd8" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 33885 } } peers { permanent_uuid: "5c449438d1ea448f8c0518eac20b8194" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 36107 } }
I20260923 15:04:21.020013 27979 ts_tablet_manager.cc:1428] T 4db10a4e0b6a404f94257b46eb1ac368 P f6496d7f1e484285ba207e9b3f30edd8: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260923 15:04:21.020447 28118 tablet_bootstrap.cc:492] T 4db10a4e0b6a404f94257b46eb1ac368 P 5c449438d1ea448f8c0518eac20b8194: No bootstrap required, opened a new log
I20260923 15:04:21.020502 28118 ts_tablet_manager.cc:1397] T 4db10a4e0b6a404f94257b46eb1ac368 P 5c449438d1ea448f8c0518eac20b8194: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 15:04:21.020656 28118 raft_consensus.cc:348] T 4db10a4e0b6a404f94257b46eb1ac368 P 5c449438d1ea448f8c0518eac20b8194 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53f9c19a5c7449faa9b3897191c6cfc6" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 40593 } } peers { permanent_uuid: "f6496d7f1e484285ba207e9b3f30edd8" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 33885 } } peers { permanent_uuid: "5c449438d1ea448f8c0518eac20b8194" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 36107 } }
I20260923 15:04:21.020735 28118 raft_consensus.cc:374] T 4db10a4e0b6a404f94257b46eb1ac368 P 5c449438d1ea448f8c0518eac20b8194 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:04:21.020793 28118 raft_consensus.cc:729] T 4db10a4e0b6a404f94257b46eb1ac368 P 5c449438d1ea448f8c0518eac20b8194 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 5c449438d1ea448f8c0518eac20b8194, State: Initialized, Role: FOLLOWER
I20260923 15:04:21.020856 28118 consensus_queue.cc:260] T 4db10a4e0b6a404f94257b46eb1ac368 P 5c449438d1ea448f8c0518eac20b8194 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53f9c19a5c7449faa9b3897191c6cfc6" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 40593 } } peers { permanent_uuid: "f6496d7f1e484285ba207e9b3f30edd8" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 33885 } } peers { permanent_uuid: "5c449438d1ea448f8c0518eac20b8194" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 36107 } }
I20260923 15:04:21.020949 28118 ts_tablet_manager.cc:1428] T 4db10a4e0b6a404f94257b46eb1ac368 P 5c449438d1ea448f8c0518eac20b8194: Time spent starting tablet: real 0.000s user 0.000s sys 0.000s
I20260923 15:04:21.022629 28094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:21.016496 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.131:40593 (local address 127.0.0.1:46524)
0923 15:04:21.016560 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0923 15:04:21.016572 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 15:04:21.016611 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:04:21.018760 (+ 2149us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:04:21.018762 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:04:21.018766 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:04:21.018820 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:04:21.018821 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:04:21.020293 (+ 1472us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:04:21.020294 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:04:21.021431 (+ 1137us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:04:21.021435 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:04:21.022374 (+ 939us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:04:21.022375 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:04:21.022448 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:04:21.022452 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 15:04:21.022509 (+ 57us) client_negotiation.cc:240] Negotiation successful
0923 15:04:21.022551 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":44}
I20260923 15:04:21.022644 28368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:21.016704 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46524 (local address 127.13.75.131:40593)
0923 15:04:21.018575 (+ 1871us) server_negotiation.cc:206] Beginning negotiation
0923 15:04:21.018581 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 15:04:21.018592 (+ 11us) server_negotiation.cc:407] Connection header received
0923 15:04:21.018633 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:04:21.018636 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:04:21.018653 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:04:21.018748 (+ 95us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:04:21.018911 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:04:21.020161 (+ 1250us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:04:21.021521 (+ 1360us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:04:21.022277 (+ 756us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:04:21.022372 (+ 95us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:04:21.022436 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0923 15:04:21.022524 (+ 88us) server_negotiation.cc:299] Negotiation successful
0923 15:04:21.022561 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":1827,"thread_start_us":111,"threads_started":1}
I20260923 15:04:21.023382 28296 tablet_service.cc:1467] Processing CreateTablet for tablet 4db10a4e0b6a404f94257b46eb1ac368 (DEFAULT_TABLE table=test-table [id=09b096ac9704473386a300178f335c0d]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 15:04:21.023713 28296 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 4db10a4e0b6a404f94257b46eb1ac368. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:04:21.026563 28371 tablet_bootstrap.cc:492] T 4db10a4e0b6a404f94257b46eb1ac368 P 53f9c19a5c7449faa9b3897191c6cfc6: Bootstrap starting.
I20260923 15:04:21.027267 28371 tablet_bootstrap.cc:654] T 4db10a4e0b6a404f94257b46eb1ac368 P 53f9c19a5c7449faa9b3897191c6cfc6: Neither blocks nor log segments found. Creating new log.
I20260923 15:04:21.027784 28371 log.cc:824] T 4db10a4e0b6a404f94257b46eb1ac368 P 53f9c19a5c7449faa9b3897191c6cfc6: Log is configured to *not* fsync() on all Append() calls
I20260923 15:04:21.028750 28371 tablet_bootstrap.cc:492] T 4db10a4e0b6a404f94257b46eb1ac368 P 53f9c19a5c7449faa9b3897191c6cfc6: No bootstrap required, opened a new log
I20260923 15:04:21.028834 28371 ts_tablet_manager.cc:1397] T 4db10a4e0b6a404f94257b46eb1ac368 P 53f9c19a5c7449faa9b3897191c6cfc6: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260923 15:04:21.030297 28371 raft_consensus.cc:348] T 4db10a4e0b6a404f94257b46eb1ac368 P 53f9c19a5c7449faa9b3897191c6cfc6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53f9c19a5c7449faa9b3897191c6cfc6" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 40593 } } peers { permanent_uuid: "f6496d7f1e484285ba207e9b3f30edd8" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 33885 } } peers { permanent_uuid: "5c449438d1ea448f8c0518eac20b8194" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 36107 } }
I20260923 15:04:21.030460 28371 raft_consensus.cc:374] T 4db10a4e0b6a404f94257b46eb1ac368 P 53f9c19a5c7449faa9b3897191c6cfc6 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:04:21.030510 28371 raft_consensus.cc:729] T 4db10a4e0b6a404f94257b46eb1ac368 P 53f9c19a5c7449faa9b3897191c6cfc6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 53f9c19a5c7449faa9b3897191c6cfc6, State: Initialized, Role: FOLLOWER
I20260923 15:04:21.030642 28371 consensus_queue.cc:260] T 4db10a4e0b6a404f94257b46eb1ac368 P 53f9c19a5c7449faa9b3897191c6cfc6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53f9c19a5c7449faa9b3897191c6cfc6" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 40593 } } peers { permanent_uuid: "f6496d7f1e484285ba207e9b3f30edd8" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 33885 } } peers { permanent_uuid: "5c449438d1ea448f8c0518eac20b8194" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 36107 } }
I20260923 15:04:21.031021 28371 ts_tablet_manager.cc:1428] T 4db10a4e0b6a404f94257b46eb1ac368 P 53f9c19a5c7449faa9b3897191c6cfc6: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260923 15:04:21.031067 28362 heartbeater.cc:503] Master 127.13.75.190:46259 was elected leader, sending a full tablet report...
I20260923 15:04:21.066511 28230 raft_consensus.cc:482] T 4db10a4e0b6a404f94257b46eb1ac368 P 5c449438d1ea448f8c0518eac20b8194 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 15:04:21.066682 28230 raft_consensus.cc:504] T 4db10a4e0b6a404f94257b46eb1ac368 P 5c449438d1ea448f8c0518eac20b8194 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53f9c19a5c7449faa9b3897191c6cfc6" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 40593 } } peers { permanent_uuid: "f6496d7f1e484285ba207e9b3f30edd8" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 33885 } } peers { permanent_uuid: "5c449438d1ea448f8c0518eac20b8194" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 36107 } }
I20260923 15:04:21.067067 28230 leader_election.cc:288] T 4db10a4e0b6a404f94257b46eb1ac368 P 5c449438d1ea448f8c0518eac20b8194 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 53f9c19a5c7449faa9b3897191c6cfc6 (127.13.75.131:40593), f6496d7f1e484285ba207e9b3f30edd8 (127.13.75.129:33885)
I20260923 15:04:21.069927 28095 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:21.067371 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.130:59729 (local address 127.13.75.129:33885)
0923 15:04:21.067423 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0923 15:04:21.067425 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:04:21.067445 (+ 20us) server_negotiation.cc:407] Connection header received
0923 15:04:21.067471 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:04:21.067472 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:04:21.067483 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:04:21.067549 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:04:21.067706 (+ 157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:04:21.068192 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:04:21.069113 (+ 921us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:04:21.069555 (+ 442us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:04:21.069635 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:04:21.069645 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0923 15:04:21.069825 (+ 180us) server_negotiation.cc:299] Negotiation successful
0923 15:04:21.069863 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260923 15:04:21.069941 28228 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:21.067258 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.129:33885 (local address 127.13.75.130:59729)
0923 15:04:21.067315 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0923 15:04:21.067327 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 15:04:21.067385 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:04:21.067554 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:04:21.067555 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:04:21.067556 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:04:21.067630 (+ 74us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:04:21.067632 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:04:21.068283 (+ 651us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:04:21.068284 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:04:21.069015 (+ 731us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:04:21.069017 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:04:21.069631 (+ 614us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:04:21.069632 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:04:21.069718 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:04:21.069745 (+ 27us) client_negotiation.cc:769] Sending connection context
0923 15:04:21.069805 (+ 60us) client_negotiation.cc:240] Negotiation successful
0923 15:04:21.069846 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260923 15:04:21.070394 28374 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:21.067261 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.131:40593 (local address 127.13.75.130:52253)
0923 15:04:21.067527 (+ 266us) negotiation.cc:107] Waiting for socket to connect
0923 15:04:21.067543 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0923 15:04:21.067619 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:04:21.067745 (+ 126us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:04:21.067745 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:04:21.067747 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:04:21.067822 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:04:21.067829 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:04:21.068634 (+ 805us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:04:21.068636 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:04:21.069491 (+ 855us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:04:21.069494 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:04:21.070180 (+ 686us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:04:21.070181 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:04:21.070254 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:04:21.070257 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 15:04:21.070306 (+ 49us) client_negotiation.cc:240] Negotiation successful
0923 15:04:21.070332 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":249,"mutex_wait_us":75,"thread_start_us":136,"threads_started":1}
I20260923 15:04:21.070472 28368 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:21.067339 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.130:52253 (local address 127.13.75.131:40593)
0923 15:04:21.067398 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0923 15:04:21.067400 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:04:21.067606 (+ 206us) server_negotiation.cc:407] Connection header received
0923 15:04:21.067676 (+ 70us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:04:21.067677 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:04:21.067686 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:04:21.067741 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:04:21.067900 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:04:21.068388 (+ 488us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:04:21.069587 (+ 1199us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:04:21.070073 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:04:21.070147 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:04:21.070154 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0923 15:04:21.070336 (+ 182us) server_negotiation.cc:299] Negotiation successful
0923 15:04:21.070376 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260923 15:04:21.070768 28316 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4db10a4e0b6a404f94257b46eb1ac368" candidate_uuid: "5c449438d1ea448f8c0518eac20b8194" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53f9c19a5c7449faa9b3897191c6cfc6" is_pre_election: true
I20260923 15:04:21.070259 28041 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4db10a4e0b6a404f94257b46eb1ac368" candidate_uuid: "5c449438d1ea448f8c0518eac20b8194" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6496d7f1e484285ba207e9b3f30edd8" is_pre_election: true
I20260923 15:04:21.070971 28041 raft_consensus.cc:2454] T 4db10a4e0b6a404f94257b46eb1ac368 P f6496d7f1e484285ba207e9b3f30edd8 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c449438d1ea448f8c0518eac20b8194 in term 0.
I20260923 15:04:21.070971 28316 raft_consensus.cc:2454] T 4db10a4e0b6a404f94257b46eb1ac368 P 53f9c19a5c7449faa9b3897191c6cfc6 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 5c449438d1ea448f8c0518eac20b8194 in term 0.
I20260923 15:04:21.071192 28113 leader_election.cc:302] T 4db10a4e0b6a404f94257b46eb1ac368 P 5c449438d1ea448f8c0518eac20b8194 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 5c449438d1ea448f8c0518eac20b8194, f6496d7f1e484285ba207e9b3f30edd8; no voters:
I20260923 15:04:21.071323 28230 raft_consensus.cc:2781] T 4db10a4e0b6a404f94257b46eb1ac368 P 5c449438d1ea448f8c0518eac20b8194 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 15:04:21.071380 28230 raft_consensus.cc:482] T 4db10a4e0b6a404f94257b46eb1ac368 P 5c449438d1ea448f8c0518eac20b8194 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 15:04:21.071403 28230 raft_consensus.cc:3037] T 4db10a4e0b6a404f94257b46eb1ac368 P 5c449438d1ea448f8c0518eac20b8194 [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:04:21.072113 28230 raft_consensus.cc:504] T 4db10a4e0b6a404f94257b46eb1ac368 P 5c449438d1ea448f8c0518eac20b8194 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53f9c19a5c7449faa9b3897191c6cfc6" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 40593 } } peers { permanent_uuid: "f6496d7f1e484285ba207e9b3f30edd8" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 33885 } } peers { permanent_uuid: "5c449438d1ea448f8c0518eac20b8194" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 36107 } }
I20260923 15:04:21.072302 28230 leader_election.cc:288] T 4db10a4e0b6a404f94257b46eb1ac368 P 5c449438d1ea448f8c0518eac20b8194 [CANDIDATE]: Term 1 election: Requested vote from peers 53f9c19a5c7449faa9b3897191c6cfc6 (127.13.75.131:40593), f6496d7f1e484285ba207e9b3f30edd8 (127.13.75.129:33885)
I20260923 15:04:21.072456 28316 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4db10a4e0b6a404f94257b46eb1ac368" candidate_uuid: "5c449438d1ea448f8c0518eac20b8194" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "53f9c19a5c7449faa9b3897191c6cfc6"
I20260923 15:04:21.072454 28041 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "4db10a4e0b6a404f94257b46eb1ac368" candidate_uuid: "5c449438d1ea448f8c0518eac20b8194" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f6496d7f1e484285ba207e9b3f30edd8"
I20260923 15:04:21.072532 28316 raft_consensus.cc:3037] T 4db10a4e0b6a404f94257b46eb1ac368 P 53f9c19a5c7449faa9b3897191c6cfc6 [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:04:21.072553 28041 raft_consensus.cc:3037] T 4db10a4e0b6a404f94257b46eb1ac368 P f6496d7f1e484285ba207e9b3f30edd8 [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:04:21.073230 28316 raft_consensus.cc:2454] T 4db10a4e0b6a404f94257b46eb1ac368 P 53f9c19a5c7449faa9b3897191c6cfc6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c449438d1ea448f8c0518eac20b8194 in term 1.
I20260923 15:04:21.073333 28041 raft_consensus.cc:2454] T 4db10a4e0b6a404f94257b46eb1ac368 P f6496d7f1e484285ba207e9b3f30edd8 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 5c449438d1ea448f8c0518eac20b8194 in term 1.
I20260923 15:04:21.073418 28112 leader_election.cc:302] T 4db10a4e0b6a404f94257b46eb1ac368 P 5c449438d1ea448f8c0518eac20b8194 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 53f9c19a5c7449faa9b3897191c6cfc6, 5c449438d1ea448f8c0518eac20b8194; no voters:
I20260923 15:04:21.073523 28230 raft_consensus.cc:2781] T 4db10a4e0b6a404f94257b46eb1ac368 P 5c449438d1ea448f8c0518eac20b8194 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 15:04:21.073704 28230 raft_consensus.cc:686] T 4db10a4e0b6a404f94257b46eb1ac368 P 5c449438d1ea448f8c0518eac20b8194 [term 1 LEADER]: Becoming Leader. State: Replica: 5c449438d1ea448f8c0518eac20b8194, State: Running, Role: LEADER
I20260923 15:04:21.073796 28230 consensus_queue.cc:237] T 4db10a4e0b6a404f94257b46eb1ac368 P 5c449438d1ea448f8c0518eac20b8194 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53f9c19a5c7449faa9b3897191c6cfc6" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 40593 } } peers { permanent_uuid: "f6496d7f1e484285ba207e9b3f30edd8" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 33885 } } peers { permanent_uuid: "5c449438d1ea448f8c0518eac20b8194" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 36107 } }
I20260923 15:04:21.074620 27898 catalog_manager.cc:5495] T 4db10a4e0b6a404f94257b46eb1ac368 P 5c449438d1ea448f8c0518eac20b8194 reported cstate change: term changed from 0 to 1, leader changed from <none> to 5c449438d1ea448f8c0518eac20b8194 (127.13.75.130). New cstate: current_term: 1 leader_uuid: "5c449438d1ea448f8c0518eac20b8194" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53f9c19a5c7449faa9b3897191c6cfc6" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 40593 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "f6496d7f1e484285ba207e9b3f30edd8" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 33885 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "5c449438d1ea448f8c0518eac20b8194" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 36107 } health_report { overall_health: HEALTHY } } }
I20260923 15:04:21.095891 28232 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:21.092950 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37610 (local address 127.13.75.130:36107)
0923 15:04:21.093023 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0923 15:04:21.093025 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:04:21.093036 (+ 11us) server_negotiation.cc:407] Connection header received
0923 15:04:21.093084 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:04:21.093085 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:04:21.093104 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:04:21.093184 (+ 80us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 15:04:21.093294 (+ 110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:04:21.093786 (+ 492us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:04:21.094794 (+ 1008us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:04:21.095214 (+ 420us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:04:21.095314 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:04:21.095532 (+ 218us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 15:04:21.095637 (+ 105us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 15:04:21.095721 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0923 15:04:21.095762 (+ 41us) server_negotiation.cc:299] Negotiation successful
0923 15:04:21.095797 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260923 15:04:21.097553 28041 raft_consensus.cc:1261] T 4db10a4e0b6a404f94257b46eb1ac368 P f6496d7f1e484285ba207e9b3f30edd8 [term 1 FOLLOWER]: Refusing update from remote peer 5c449438d1ea448f8c0518eac20b8194: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 15:04:21.097577 28316 raft_consensus.cc:1261] T 4db10a4e0b6a404f94257b46eb1ac368 P 53f9c19a5c7449faa9b3897191c6cfc6 [term 1 FOLLOWER]: Refusing update from remote peer 5c449438d1ea448f8c0518eac20b8194: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 15:04:21.097980 28375 consensus_queue.cc:1035] T 4db10a4e0b6a404f94257b46eb1ac368 P 5c449438d1ea448f8c0518eac20b8194 [LEADER]: Connected to new peer: Peer: permanent_uuid: "53f9c19a5c7449faa9b3897191c6cfc6" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 40593 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 15:04:21.098121 28230 consensus_queue.cc:1035] T 4db10a4e0b6a404f94257b46eb1ac368 P 5c449438d1ea448f8c0518eac20b8194 [LEADER]: Connected to new peer: Peer: permanent_uuid: "f6496d7f1e484285ba207e9b3f30edd8" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 33885 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 15:04:21.100664 28379 mvcc.cc:205] Tried to move back new op lower bound from 7332560327052693504 to 7332560326958919680. Current Snapshot: MvccSnapshot[applied={T|T < 7332560327052693504}]
I20260923 15:04:21.100711 28380 mvcc.cc:205] Tried to move back new op lower bound from 7332560327052693504 to 7332560326958919680. Current Snapshot: MvccSnapshot[applied={T|T < 7332560327052693504}]
I20260923 15:04:21.102985 27898 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin'} at 127.0.0.1:34066:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 15:04:21.103080 27898 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin'} at 127.0.0.1:34066:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 15:04:21.103849 27898 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 00499138dec94c0d888a87ea532d713f: Sending DeleteTablet for 3 replicas of tablet 4db10a4e0b6a404f94257b46eb1ac368
I20260923 15:04:21.104146 28017 tablet_service.cc:1514] Processing DeleteTablet for tablet 4db10a4e0b6a404f94257b46eb1ac368 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 15:04:21 UTC) from {username='kudu', principal='kudu/127.13.75.190@KRBTEST.COM'} at 127.0.0.1:51294
I20260923 15:04:21.104156 28160 tablet_service.cc:1514] Processing DeleteTablet for tablet 4db10a4e0b6a404f94257b46eb1ac368 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 15:04:21 UTC) from {username='kudu', principal='kudu/127.13.75.190@KRBTEST.COM'} at 127.0.0.1:37608
I20260923 15:04:21.104156 28296 tablet_service.cc:1514] Processing DeleteTablet for tablet 4db10a4e0b6a404f94257b46eb1ac368 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 15:04:21 UTC) from {username='kudu', principal='kudu/127.13.75.190@KRBTEST.COM'} at 127.0.0.1:46524
I20260923 15:04:21.104259 28096 tablet_replica.cc:330] T 4db10a4e0b6a404f94257b46eb1ac368 P f6496d7f1e484285ba207e9b3f30edd8: stopping tablet replica
I20260923 15:04:21.104319 28096 raft_consensus.cc:2229] T 4db10a4e0b6a404f94257b46eb1ac368 P f6496d7f1e484285ba207e9b3f30edd8 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 15:04:21.104341 28233 tablet_replica.cc:330] T 4db10a4e0b6a404f94257b46eb1ac368 P 5c449438d1ea448f8c0518eac20b8194: stopping tablet replica
I20260923 15:04:21.104375 28096 raft_consensus.cc:2258] T 4db10a4e0b6a404f94257b46eb1ac368 P f6496d7f1e484285ba207e9b3f30edd8 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 15:04:21.104399 28233 raft_consensus.cc:2229] T 4db10a4e0b6a404f94257b46eb1ac368 P 5c449438d1ea448f8c0518eac20b8194 [term 1 LEADER]: Raft consensus shutting down.
I20260923 15:04:21.104403 13614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKC4soy/build/release/bin/kudu with pid 27937
I20260923 15:04:21.104480 28233 raft_consensus.cc:2258] T 4db10a4e0b6a404f94257b46eb1ac368 P 5c449438d1ea448f8c0518eac20b8194 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 15:04:21.104501 28388 tablet_replica.cc:330] T 4db10a4e0b6a404f94257b46eb1ac368 P 53f9c19a5c7449faa9b3897191c6cfc6: stopping tablet replica
I20260923 15:04:21.104596 28388 raft_consensus.cc:2229] T 4db10a4e0b6a404f94257b46eb1ac368 P 53f9c19a5c7449faa9b3897191c6cfc6 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 15:04:21.104699 28388 raft_consensus.cc:2258] T 4db10a4e0b6a404f94257b46eb1ac368 P 53f9c19a5c7449faa9b3897191c6cfc6 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 15:04:21.104803 28233 ts_tablet_manager.cc:1905] T 4db10a4e0b6a404f94257b46eb1ac368 P 5c449438d1ea448f8c0518eac20b8194: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 15:04:21.106264 28388 ts_tablet_manager.cc:1905] T 4db10a4e0b6a404f94257b46eb1ac368 P 53f9c19a5c7449faa9b3897191c6cfc6: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 15:04:21.106321 28233 ts_tablet_manager.cc:1918] T 4db10a4e0b6a404f94257b46eb1ac368 P 5c449438d1ea448f8c0518eac20b8194: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 15:04:21.106372 28233 log.cc:1196] T 4db10a4e0b6a404f94257b46eb1ac368 P 5c449438d1ea448f8c0518eac20b8194: Deleting WAL directory at /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/minicluster-data/ts-1/wal/wals/4db10a4e0b6a404f94257b46eb1ac368
I20260923 15:04:21.106588 28233 ts_tablet_manager.cc:1939] T 4db10a4e0b6a404f94257b46eb1ac368 P 5c449438d1ea448f8c0518eac20b8194: Deleting consensus metadata
I20260923 15:04:21.106926 27882 catalog_manager.cc:4841] TS 5c449438d1ea448f8c0518eac20b8194 (127.13.75.130:36107): tablet 4db10a4e0b6a404f94257b46eb1ac368 (table test-table [id=09b096ac9704473386a300178f335c0d]) successfully deleted
I20260923 15:04:21.108225 28388 ts_tablet_manager.cc:1918] T 4db10a4e0b6a404f94257b46eb1ac368 P 53f9c19a5c7449faa9b3897191c6cfc6: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 15:04:21.108305 28388 log.cc:1196] T 4db10a4e0b6a404f94257b46eb1ac368 P 53f9c19a5c7449faa9b3897191c6cfc6: Deleting WAL directory at /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/minicluster-data/ts-2/wal/wals/4db10a4e0b6a404f94257b46eb1ac368
I20260923 15:04:21.108584 28388 ts_tablet_manager.cc:1939] T 4db10a4e0b6a404f94257b46eb1ac368 P 53f9c19a5c7449faa9b3897191c6cfc6: Deleting consensus metadata
I20260923 15:04:21.109010 27882 catalog_manager.cc:4841] TS 53f9c19a5c7449faa9b3897191c6cfc6 (127.13.75.131:40593): tablet 4db10a4e0b6a404f94257b46eb1ac368 (table test-table [id=09b096ac9704473386a300178f335c0d]) successfully deleted
W20260923 15:04:21.111441 27883 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.75.129:33885 (error 108)
I20260923 15:04:21.111630 13614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKC4soy/build/release/bin/kudu with pid 28092
I20260923 15:04:21.112443 28094 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:21.112301 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.129:33885 (local address 127.0.0.1:51306)
0923 15:04:21.112352 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0923 15:04:21.112393 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.129:33885: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260923 15:04:21.112556 27883 catalog_manager.cc:4584] TS f6496d7f1e484285ba207e9b3f30edd8 (127.13.75.129:33885): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 4db10a4e0b6a404f94257b46eb1ac368: Network error: Client connection negotiation failed: client connection to 127.13.75.129:33885: connect: Connection refused (error 111)
I20260923 15:04:21.116602 13614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKC4soy/build/release/bin/kudu with pid 28231
I20260923 15:04:21.121923 13614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKC4soy/build/release/bin/kudu with pid 27865
I20260923 15:04:21.128121 13614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKC4soy/build/release/bin/kudu
/tmp/dist-test-taskKC4soy/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.75.190:46259
--webserver_interface=127.13.75.190
--webserver_port=38245
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.190.keytab
--principal=kudu/127.13.75.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.75.148:44253
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.75.190:46259
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:04:21.211936 28390 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:04:21.212208 28390 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:04:21.212251 28390 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:04:21.213627 28390 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 15:04:21.213689 28390 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:04:21.213712 28390 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:04:21.213739 28390 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 15:04:21.213770 28390 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 15:04:21.215341 28390 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.75.148:44253
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.75.190:46259
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.190.keytab
--principal=kudu/127.13.75.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.75.190:46259
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.75.190
--webserver_port=38245
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28390
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:04:21.215642 28390 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:04:21.215889 28390 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:04:21.220929 28398 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:04:21.220984 28390 server_base.cc:1034] running on GCE node
W20260923 15:04:21.220932 28395 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:04:21.221014 28396 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:04:21.221431 28390 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:04:21.221773 28390 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:04:21.223001 28390 hybrid_clock.cc:648] HybridClock initialized: now 1790175861222972 us; error 38 us; skew 500 ppm
Sep 23 15:04:21 dist-test-slave-cnb0 krb5kdc[27318](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175861, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:04:21.225965 28390 init.cc:377] Logged in from keytab as kudu/127.13.75.190@KRBTEST.COM (short username kudu)
I20260923 15:04:21.226538 28390 webserver.cc:460] Webserver started at http://127.13.75.190:38245/ using document root <none> and password file <none>
I20260923 15:04:21.226807 28390 fs_manager.cc:359] Metadata directory not provided
I20260923 15:04:21.226888 28390 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:04:21.228555 28390 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260923 15:04:21.229435 28405 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:04:21.229646 28390 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260923 15:04:21.229786 28390 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/minicluster-data/master-0/data,/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/minicluster-data/master-0/wal
uuid: "00499138dec94c0d888a87ea532d713f"
format_stamp: "Formatted at 2026-09-23 15:04:19 on dist-test-slave-cnb0"
server_key: "397714a7c6559323a9394756f5aabe94"
server_key_iv: "2213b3efe8a8ba55a3451100dc7ef6ea"
server_key_version: "encryptionkey@0"
I20260923 15:04:21.230172 28390 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:04:21.254390 28390 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:04:21.257088 28390 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:04:21.257288 28390 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:04:21.261816 28390 rpc_server.cc:307] RPC server started. Bound to: 127.13.75.190:46259
I20260923 15:04:21.261860 28457 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.75.190:46259 every 8 connection(s)
I20260923 15:04:21.262411 28390 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/minicluster-data/master-0/data/info.pb
I20260923 15:04:21.264047 13614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKC4soy/build/release/bin/kudu as pid 28390
I20260923 15:04:21.264271 28458 sys_catalog.cc:263] Verifying existing consensus state
I20260923 15:04:21.264376 13614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKC4soy/build/release/bin/kudu
/tmp/dist-test-taskKC4soy/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.75.129:33885
--local_ip_for_outbound_sockets=127.13.75.129
--tserver_master_addrs=127.13.75.190:46259
--webserver_port=35765
--webserver_interface=127.13.75.129
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.129.keytab
--principal=kudu/127.13.75.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.75.148:44253
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260923 15:04:21.265013 28458 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 00499138dec94c0d888a87ea532d713f: Bootstrap starting.
I20260923 15:04:21.274147 28458 log.cc:824] T 00000000000000000000000000000000 P 00499138dec94c0d888a87ea532d713f: Log is configured to *not* fsync() on all Append() calls
I20260923 15:04:21.276530 28458 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 00499138dec94c0d888a87ea532d713f: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=7 ignored=0} mutations{seen=9 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260923 15:04:21.276867 28458 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 00499138dec94c0d888a87ea532d713f: Bootstrap complete.
I20260923 15:04:21.279538 28458 raft_consensus.cc:348] T 00000000000000000000000000000000 P 00499138dec94c0d888a87ea532d713f [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00499138dec94c0d888a87ea532d713f" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 46259 } }
I20260923 15:04:21.279894 28458 raft_consensus.cc:729] T 00000000000000000000000000000000 P 00499138dec94c0d888a87ea532d713f [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 00499138dec94c0d888a87ea532d713f, State: Initialized, Role: FOLLOWER
I20260923 15:04:21.280069 28458 consensus_queue.cc:260] T 00000000000000000000000000000000 P 00499138dec94c0d888a87ea532d713f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00499138dec94c0d888a87ea532d713f" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 46259 } }
I20260923 15:04:21.280153 28458 raft_consensus.cc:388] T 00000000000000000000000000000000 P 00499138dec94c0d888a87ea532d713f [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 15:04:21.280202 28458 raft_consensus.cc:482] T 00000000000000000000000000000000 P 00499138dec94c0d888a87ea532d713f [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 15:04:21.280244 28458 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 00499138dec94c0d888a87ea532d713f [term 2 FOLLOWER]: Advancing to term 3
I20260923 15:04:21.281850 28458 raft_consensus.cc:504] T 00000000000000000000000000000000 P 00499138dec94c0d888a87ea532d713f [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00499138dec94c0d888a87ea532d713f" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 46259 } }
I20260923 15:04:21.281962 28458 leader_election.cc:302] T 00000000000000000000000000000000 P 00499138dec94c0d888a87ea532d713f [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 00499138dec94c0d888a87ea532d713f; no voters:
I20260923 15:04:21.282181 28458 leader_election.cc:288] T 00000000000000000000000000000000 P 00499138dec94c0d888a87ea532d713f [CANDIDATE]: Term 3 election: Requested vote from peers
I20260923 15:04:21.282269 28462 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 00499138dec94c0d888a87ea532d713f [term 3 FOLLOWER]: Leader election won for term 3
I20260923 15:04:21.282475 28462 raft_consensus.cc:686] T 00000000000000000000000000000000 P 00499138dec94c0d888a87ea532d713f [term 3 LEADER]: Becoming Leader. State: Replica: 00499138dec94c0d888a87ea532d713f, State: Running, Role: LEADER
I20260923 15:04:21.282480 28458 sys_catalog.cc:564] T 00000000000000000000000000000000 P 00499138dec94c0d888a87ea532d713f [sys.catalog]: configured and running, proceeding with master startup.
I20260923 15:04:21.282721 28462 consensus_queue.cc:237] T 00000000000000000000000000000000 P 00499138dec94c0d888a87ea532d713f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00499138dec94c0d888a87ea532d713f" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 46259 } }
I20260923 15:04:21.283710 28463 sys_catalog.cc:455] T 00000000000000000000000000000000 P 00499138dec94c0d888a87ea532d713f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "00499138dec94c0d888a87ea532d713f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00499138dec94c0d888a87ea532d713f" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 46259 } } }
I20260923 15:04:21.283800 28463 sys_catalog.cc:458] T 00000000000000000000000000000000 P 00499138dec94c0d888a87ea532d713f [sys.catalog]: This master's current role is: LEADER
I20260923 15:04:21.284036 28464 sys_catalog.cc:455] T 00000000000000000000000000000000 P 00499138dec94c0d888a87ea532d713f [sys.catalog]: SysCatalogTable state changed. Reason: New leader 00499138dec94c0d888a87ea532d713f. Latest consensus state: current_term: 3 leader_uuid: "00499138dec94c0d888a87ea532d713f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "00499138dec94c0d888a87ea532d713f" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 46259 } } }
I20260923 15:04:21.284178 28464 sys_catalog.cc:458] T 00000000000000000000000000000000 P 00499138dec94c0d888a87ea532d713f [sys.catalog]: This master's current role is: LEADER
I20260923 15:04:21.284543 28477 catalog_manager.cc:1259] Loaded cluster ID: 42cd13b126f348e88b37a4472b55607d
I20260923 15:04:21.284624 28477 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 00499138dec94c0d888a87ea532d713f: loading cluster ID for follower catalog manager: success
W20260923 15:04:21.284917 28477 catalog_manager.cc:1576] T 00000000000000000000000000000000 P 00499138dec94c0d888a87ea532d713f: acquiring CA information for follower catalog manager: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
W20260923 15:04:21.285063 28477 catalog_manager.cc:873] Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO: failed to prepare follower catalog manager, will retry
I20260923 15:04:21.285177 28478 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 15:04:21.285655 28478 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 15:04:21.285737 28478 catalog_manager.cc:1259] Loaded cluster ID: 42cd13b126f348e88b37a4472b55607d
I20260923 15:04:21.285768 28478 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
F20260923 15:04:21.285897 28478 catalog_manager.cc:1466] Initializing Kudu internal certificate authority failed: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f9866c5d9ad google::LogMessage::Fail() at ??:0
@ 0x7f9866c6088a google::LogMessage::SendToLog() at ??:0
@ 0x7f9866c5d4de google::LogMessage::Flush() at ??:0
@ 0x7f9866c5ea19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f986eb7e0f1 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7f986eb878b9 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f9868b2b0d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f9868b20954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f98699846db start_thread at ??:0
@ 0x7f986575b61f clone at ??:0
W20260923 15:04:21.359333 28460 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:04:21.359565 28460 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:04:21.359602 28460 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:04:21.360760 28460 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:04:21.360833 28460 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:04:21.360881 28460 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.75.129
I20260923 15:04:21.362385 28460 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.75.148:44253
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.129.keytab
--principal=kudu/127.13.75.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.75.129:33885
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.75.129
--webserver_port=35765
--tserver_master_addrs=127.13.75.190:46259
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28460
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.75.129
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:04:21.362679 28460 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:04:21.362982 28460 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:04:21.368031 28497 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:04:21.368034 28494 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:04:21.368395 28460 server_base.cc:1034] running on GCE node
W20260923 15:04:21.368139 28495 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:04:21.368656 28460 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:04:21.368947 28460 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:04:21.370119 28460 hybrid_clock.cc:648] HybridClock initialized: now 1790175861370094 us; error 41 us; skew 500 ppm
Sep 23 15:04:21 dist-test-slave-cnb0 krb5kdc[27318](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175861, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:04:21.373265 28460 init.cc:377] Logged in from keytab as kudu/127.13.75.129@KRBTEST.COM (short username kudu)
I20260923 15:04:21.373852 28460 webserver.cc:460] Webserver started at http://127.13.75.129:35765/ using document root <none> and password file <none>
I20260923 15:04:21.374140 28460 fs_manager.cc:359] Metadata directory not provided
I20260923 15:04:21.374208 28460 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:04:21.376104 28460 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:04:21.376947 28504 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:04:21.377187 28460 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:04:21.377336 28460 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/minicluster-data/ts-0/wal
uuid: "f6496d7f1e484285ba207e9b3f30edd8"
format_stamp: "Formatted at 2026-09-23 15:04:19 on dist-test-slave-cnb0"
server_key: "a7714ab93270211d1d05b2a4cc05be9b"
server_key_iv: "edffafa014fff3f00c0a95d2207c6674"
server_key_version: "encryptionkey@0"
I20260923 15:04:21.377699 28460 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:04:21.388898 28460 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:04:21.391639 28460 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:04:21.391811 28460 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:04:21.392086 28460 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 15:04:21.392590 28511 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260923 15:04:21.394622 28460 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260923 15:04:21.394706 28460 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260923 15:04:21.394764 28460 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260923 15:04:21.395570 28460 ts_tablet_manager.cc:610] Registered 1 tablets
I20260923 15:04:21.395632 28460 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260923 15:04:21.395706 28511 tablet_bootstrap.cc:492] T 4db10a4e0b6a404f94257b46eb1ac368 P f6496d7f1e484285ba207e9b3f30edd8: Bootstrap starting.
I20260923 15:04:21.402520 28460 rpc_server.cc:307] RPC server started. Bound to: 127.13.75.129:33885
I20260923 15:04:21.402673 28619 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.75.129:33885 every 8 connection(s)
I20260923 15:04:21.403041 28460 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/minicluster-data/ts-0/data/info.pb
I20260923 15:04:21.407088 28511 log.cc:824] T 4db10a4e0b6a404f94257b46eb1ac368 P f6496d7f1e484285ba207e9b3f30edd8: Log is configured to *not* fsync() on all Append() calls
I20260923 15:04:21.409871 28511 tablet_bootstrap.cc:492] T 4db10a4e0b6a404f94257b46eb1ac368 P f6496d7f1e484285ba207e9b3f30edd8: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260923 15:04:21.410228 28511 tablet_bootstrap.cc:492] T 4db10a4e0b6a404f94257b46eb1ac368 P f6496d7f1e484285ba207e9b3f30edd8: Bootstrap complete.
I20260923 15:04:21.410218 13614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKC4soy/build/release/bin/kudu as pid 28460
I20260923 15:04:21.410518 13614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKC4soy/build/release/bin/kudu
/tmp/dist-test-taskKC4soy/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.75.130:36107
--local_ip_for_outbound_sockets=127.13.75.130
--tserver_master_addrs=127.13.75.190:46259
--webserver_port=39943
--webserver_interface=127.13.75.130
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.130.keytab
--principal=kudu/127.13.75.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.75.148:44253
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260923 15:04:21.411602 28511 ts_tablet_manager.cc:1397] T 4db10a4e0b6a404f94257b46eb1ac368 P f6496d7f1e484285ba207e9b3f30edd8: Time spent bootstrapping tablet: real 0.016s user 0.007s sys 0.008s
I20260923 15:04:21.413260 28511 raft_consensus.cc:348] T 4db10a4e0b6a404f94257b46eb1ac368 P f6496d7f1e484285ba207e9b3f30edd8 [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53f9c19a5c7449faa9b3897191c6cfc6" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 40593 } } peers { permanent_uuid: "f6496d7f1e484285ba207e9b3f30edd8" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 33885 } } peers { permanent_uuid: "5c449438d1ea448f8c0518eac20b8194" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 36107 } }
I20260923 15:04:21.413600 28511 raft_consensus.cc:729] T 4db10a4e0b6a404f94257b46eb1ac368 P f6496d7f1e484285ba207e9b3f30edd8 [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: f6496d7f1e484285ba207e9b3f30edd8, State: Initialized, Role: FOLLOWER
I20260923 15:04:21.413787 28511 consensus_queue.cc:260] T 4db10a4e0b6a404f94257b46eb1ac368 P f6496d7f1e484285ba207e9b3f30edd8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53f9c19a5c7449faa9b3897191c6cfc6" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 40593 } } peers { permanent_uuid: "f6496d7f1e484285ba207e9b3f30edd8" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 33885 } } peers { permanent_uuid: "5c449438d1ea448f8c0518eac20b8194" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 36107 } }
I20260923 15:04:21.414095 28511 ts_tablet_manager.cc:1428] T 4db10a4e0b6a404f94257b46eb1ac368 P f6496d7f1e484285ba207e9b3f30edd8: Time spent starting tablet: real 0.002s user 0.001s sys 0.002s
W20260923 15:04:21.478792 28622 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 15:04:21.403764 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:46259 (local address 127.13.75.129:37601)
0923 15:04:21.404080 (+ 316us) negotiation.cc:107] Waiting for socket to connect
0923 15:04:21.404113 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0923 15:04:21.404575 (+ 462us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:04:21.478545 (+ 73970us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.190:46259: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":274,"thread_start_us":72,"threads_started":1}
W20260923 15:04:21.479112 28620 heartbeater.cc:644] Failed to heartbeat to 127.13.75.190:46259 (0 consecutive failures): Network error: Failed to ping master at 127.13.75.190:46259: Client connection negotiation failed: client connection to 127.13.75.190:46259: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260923 15:04:21.479528 28622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:21.479406 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:46259 (local address 127.13.75.129:47473)
0923 15:04:21.479463 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0923 15:04:21.479509 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.190:46259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260923 15:04:21.479919 28622 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:21.479814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:46259 (local address 127.13.75.129:48069)
0923 15:04:21.479866 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0923 15:04:21.479893 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.190:46259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
W20260923 15:04:21.480052 28620 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260923 15:04:21.494771 28624 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:04:21.495069 28624 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:04:21.495123 28624 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:04:21.496439 28624 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:04:21.496508 28624 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:04:21.496557 28624 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.75.130
I20260923 15:04:21.498202 28624 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.75.148:44253
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.130.keytab
--principal=kudu/127.13.75.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.75.130:36107
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.75.130
--webserver_port=39943
--tserver_master_addrs=127.13.75.190:46259
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28624
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.75.130
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:04:21.498498 28624 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:04:21.498762 28624 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:04:21.504225 28630 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:04:21.504331 28624 server_base.cc:1034] running on GCE node
W20260923 15:04:21.504249 28633 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:04:21.504436 28631 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:04:21.504696 28624 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:04:21.504971 28624 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:04:21.506146 28624 hybrid_clock.cc:648] HybridClock initialized: now 1790175861506131 us; error 28 us; skew 500 ppm
Sep 23 15:04:21 dist-test-slave-cnb0 krb5kdc[27318](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175861, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:04:21.508736 28624 init.cc:377] Logged in from keytab as kudu/127.13.75.130@KRBTEST.COM (short username kudu)
I20260923 15:04:21.509167 28624 webserver.cc:460] Webserver started at http://127.13.75.130:39943/ using document root <none> and password file <none>
I20260923 15:04:21.509393 28624 fs_manager.cc:359] Metadata directory not provided
I20260923 15:04:21.509438 28624 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:04:21.510769 28624 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:04:21.511538 28640 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:04:21.511791 28624 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260923 15:04:21.511893 28624 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/minicluster-data/ts-1/wal
uuid: "5c449438d1ea448f8c0518eac20b8194"
format_stamp: "Formatted at 2026-09-23 15:04:20 on dist-test-slave-cnb0"
server_key: "0751eb9a74520531b5cb362090d40b54"
server_key_iv: "d8090d7eb65569f9b73c8389b6be7e1a"
server_key_version: "encryptionkey@0"
I20260923 15:04:21.512245 28624 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:04:21.521462 28624 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:04:21.523375 28624 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:04:21.523478 28624 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:04:21.523644 28624 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 15:04:21.523933 28624 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 15:04:21.523972 28624 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 15:04:21.523993 28624 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 15:04:21.524035 28624 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 15:04:21.530058 28624 rpc_server.cc:307] RPC server started. Bound to: 127.13.75.130:36107
I20260923 15:04:21.530107 28753 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.75.130:36107 every 8 connection(s)
I20260923 15:04:21.530467 28624 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/minicluster-data/ts-1/data/info.pb
I20260923 15:04:21.531629 28756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:21.531231 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:46259 (local address 127.13.75.130:60699)
0923 15:04:21.531409 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0923 15:04:21.531486 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.190:46259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":135,"thread_start_us":74,"threads_started":1}
W20260923 15:04:21.532088 28754 heartbeater.cc:644] Failed to heartbeat to 127.13.75.190:46259 (0 consecutive failures): Network error: Failed to ping master at 127.13.75.190:46259: Client connection negotiation failed: client connection to 127.13.75.190:46259: connect: Connection refused (error 111)
I20260923 15:04:21.532387 28756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:21.532288 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:46259 (local address 127.13.75.130:60399)
0923 15:04:21.532332 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0923 15:04:21.532361 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.190:46259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
I20260923 15:04:21.532689 28756 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:21.532611 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:46259 (local address 127.13.75.130:38171)
0923 15:04:21.532651 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0923 15:04:21.532674 (+ 23us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.190:46259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260923 15:04:21.532886 28754 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260923 15:04:21.536541 13614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKC4soy/build/release/bin/kudu as pid 28624
I20260923 15:04:21.536808 13614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKC4soy/build/release/bin/kudu
/tmp/dist-test-taskKC4soy/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.75.131:40593
--local_ip_for_outbound_sockets=127.13.75.131
--tserver_master_addrs=127.13.75.190:46259
--webserver_port=43299
--webserver_interface=127.13.75.131
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.131.keytab
--principal=kudu/127.13.75.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.75.148:44253
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:04:21.619975 28757 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:04:21.620195 28757 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:04:21.620220 28757 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:04:21.621423 28757 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:04:21.621466 28757 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:04:21.621495 28757 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.75.131
I20260923 15:04:21.623029 28757 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.75.148:44253
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.131.keytab
--principal=kudu/127.13.75.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.75.131:40593
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.75.131
--webserver_port=43299
--tserver_master_addrs=127.13.75.190:46259
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28757
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.75.131
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:04:21.623283 28757 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:04:21.623536 28757 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:04:21.628432 28762 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:04:21.628508 28765 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:04:21.628647 28757 server_base.cc:1034] running on GCE node
W20260923 15:04:21.628505 28763 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:04:21.628999 28757 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:04:21.629212 28757 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:04:21.630419 28757 hybrid_clock.cc:648] HybridClock initialized: now 1790175861630398 us; error 37 us; skew 500 ppm
Sep 23 15:04:21 dist-test-slave-cnb0 krb5kdc[27318](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175861, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:04:21.632877 28757 init.cc:377] Logged in from keytab as kudu/127.13.75.131@KRBTEST.COM (short username kudu)
I20260923 15:04:21.633342 28757 webserver.cc:460] Webserver started at http://127.13.75.131:43299/ using document root <none> and password file <none>
I20260923 15:04:21.633589 28757 fs_manager.cc:359] Metadata directory not provided
I20260923 15:04:21.633656 28757 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:04:21.635012 28757 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.000s sys 0.001s
I20260923 15:04:21.635759 28772 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:04:21.635963 28757 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260923 15:04:21.636090 28757 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/minicluster-data/ts-2/wal
uuid: "53f9c19a5c7449faa9b3897191c6cfc6"
format_stamp: "Formatted at 2026-09-23 15:04:20 on dist-test-slave-cnb0"
server_key: "36bc0810866c35e78372225ca65d4ece"
server_key_iv: "3b934cb902d001ecaffb886f3a50f220"
server_key_version: "encryptionkey@0"
I20260923 15:04:21.636574 28757 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:04:21.648938 28757 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:04:21.651289 28757 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:04:21.651408 28757 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:04:21.651597 28757 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 15:04:21.651907 28757 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 15:04:21.651942 28757 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 15:04:21.651963 28757 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 15:04:21.652007 28757 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 15:04:21.657785 28757 rpc_server.cc:307] RPC server started. Bound to: 127.13.75.131:40593
I20260923 15:04:21.657868 28885 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.75.131:40593 every 8 connection(s)
I20260923 15:04:21.658195 28757 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedIPKICert.1790175691491353-13614-0/minicluster-data/ts-2/data/info.pb
I20260923 15:04:21.659361 28888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:21.658927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:46259 (local address 127.13.75.131:45353)
0923 15:04:21.659066 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0923 15:04:21.659171 (+ 105us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.190:46259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":101,"thread_start_us":73,"threads_started":1}
W20260923 15:04:21.660045 28886 heartbeater.cc:644] Failed to heartbeat to 127.13.75.190:46259 (0 consecutive failures): Network error: Failed to ping master at 127.13.75.190:46259: Client connection negotiation failed: client connection to 127.13.75.190:46259: connect: Connection refused (error 111)
I20260923 15:04:21.660374 28888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:21.660263 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:46259 (local address 127.13.75.131:33137)
0923 15:04:21.660313 (+ 50us) negotiation.cc:107] Waiting for socket to connect
0923 15:04:21.660345 (+ 32us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.190:46259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":16}
I20260923 15:04:21.660768 28888 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:21.660670 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:46259 (local address 127.13.75.131:38623)
0923 15:04:21.660698 (+ 28us) negotiation.cc:107] Waiting for socket to connect
0923 15:04:21.660741 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.190:46259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":14}
W20260923 15:04:21.660888 28886 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260923 15:04:21.662027 13614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKC4soy/build/release/bin/kudu as pid 28757
I20260923 15:04:22.480661 28890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:22.480407 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:46259 (local address 127.13.75.129:57627)
0923 15:04:22.480557 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0923 15:04:22.480600 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.190:46259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":65,"threads_started":1}
I20260923 15:04:22.533569 28891 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:22.533325 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:46259 (local address 127.13.75.130:35653)
0923 15:04:22.533472 (+ 147us) negotiation.cc:107] Waiting for socket to connect
0923 15:04:22.533511 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.190:46259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":56,"threads_started":1}
I20260923 15:04:22.661442 28892 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:22.661235 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:46259 (local address 127.13.75.131:55649)
0923 15:04:22.661347 (+ 112us) negotiation.cc:107] Waiting for socket to connect
0923 15:04:22.661385 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.190:46259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":95,"thread_start_us":56,"threads_started":1}
I20260923 15:04:22.791906 28893 raft_consensus.cc:482] T 4db10a4e0b6a404f94257b46eb1ac368 P f6496d7f1e484285ba207e9b3f30edd8 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 15:04:22.792032 28893 raft_consensus.cc:504] T 4db10a4e0b6a404f94257b46eb1ac368 P f6496d7f1e484285ba207e9b3f30edd8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53f9c19a5c7449faa9b3897191c6cfc6" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 40593 } } peers { permanent_uuid: "f6496d7f1e484285ba207e9b3f30edd8" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 33885 } } peers { permanent_uuid: "5c449438d1ea448f8c0518eac20b8194" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 36107 } }
I20260923 15:04:22.792441 28893 leader_election.cc:288] T 4db10a4e0b6a404f94257b46eb1ac368 P f6496d7f1e484285ba207e9b3f30edd8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 53f9c19a5c7449faa9b3897191c6cfc6 (127.13.75.131:40593), 5c449438d1ea448f8c0518eac20b8194 (127.13.75.130:36107)
Sep 23 15:04:22 dist-test-slave-cnb0 krb5kdc[27318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175861, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.129@KRBTEST.COM for kudu/127.13.75.130@KRBTEST.COM
Sep 23 15:04:22 dist-test-slave-cnb0 krb5kdc[27318](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175861, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.129@KRBTEST.COM for kudu/127.13.75.131@KRBTEST.COM
I20260923 15:04:22.812228 28894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:22.792870 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.130:36107 (local address 127.13.75.129:46575)
0923 15:04:22.792977 (+ 107us) negotiation.cc:107] Waiting for socket to connect
0923 15:04:22.792988 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 15:04:22.793043 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:04:22.794023 (+ 980us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:04:22.794026 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:04:22.794546 (+ 520us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:04:22.794816 (+ 270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:04:22.794824 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:04:22.796588 (+ 1764us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:04:22.796590 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:04:22.797944 (+ 1354us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:04:22.797947 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:04:22.798870 (+ 923us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:04:22.798871 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:04:22.799073 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:04:22.805915 (+ 6842us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:04:22.805932 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:04:22.808098 (+ 2166us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:04:22.810421 (+ 2323us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:04:22.810431 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:04:22.810433 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:04:22.810803 (+ 370us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:04:22.811113 (+ 310us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:04:22.811114 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:04:22.811114 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:04:22.811242 (+ 128us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:04:22.811650 (+ 408us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:04:22.811657 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:04:22.811804 (+ 147us) client_negotiation.cc:769] Sending connection context
0923 15:04:22.811917 (+ 113us) client_negotiation.cc:240] Negotiation successful
0923 15:04:22.812126 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":95,"mutex_wait_us":78,"thread_start_us":54,"threads_started":1}
I20260923 15:04:22.812258 28896 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:22.792961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.129:46575 (local address 127.13.75.130:36107)
0923 15:04:22.793104 (+ 143us) server_negotiation.cc:206] Beginning negotiation
0923 15:04:22.793112 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0923 15:04:22.793140 (+ 28us) server_negotiation.cc:407] Connection header received
0923 15:04:22.793677 (+ 537us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:04:22.793680 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:04:22.793740 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:04:22.793913 (+ 173us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:04:22.795000 (+ 1087us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:04:22.796411 (+ 1411us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:04:22.798068 (+ 1657us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:04:22.798735 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:04:22.798860 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:04:22.808311 (+ 9451us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:04:22.808319 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:04:22.808319 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:04:22.808335 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:04:22.810312 (+ 1977us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:04:22.810910 (+ 598us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:04:22.810915 (+ 5us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:04:22.810915 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:04:22.811030 (+ 115us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:04:22.811316 (+ 286us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:04:22.811317 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:04:22.811317 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:04:22.811576 (+ 259us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:04:22.811663 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0923 15:04:22.811975 (+ 312us) server_negotiation.cc:299] Negotiation successful
0923 15:04:22.812118 (+ 143us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102,"thread_start_us":62,"threads_started":1}
I20260923 15:04:22.812280 28895 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:22.792903 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.129:60939 (local address 127.13.75.131:40593)
0923 15:04:22.793038 (+ 135us) server_negotiation.cc:206] Beginning negotiation
0923 15:04:22.793052 (+ 14us) server_negotiation.cc:399] Waiting for connection header
0923 15:04:22.793087 (+ 35us) server_negotiation.cc:407] Connection header received
0923 15:04:22.793678 (+ 591us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:04:22.793680 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:04:22.793740 (+ 60us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:04:22.793919 (+ 179us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:04:22.795000 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:04:22.796431 (+ 1431us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:04:22.798069 (+ 1638us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:04:22.798736 (+ 667us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:04:22.798860 (+ 124us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:04:22.808685 (+ 9825us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:04:22.808690 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:04:22.808691 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:04:22.808702 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:04:22.810469 (+ 1767us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:04:22.810939 (+ 470us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:04:22.810942 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:04:22.810942 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:04:22.811037 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:04:22.811319 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:04:22.811320 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:04:22.811320 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:04:22.811583 (+ 263us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:04:22.811683 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0923 15:04:22.811974 (+ 291us) server_negotiation.cc:299] Negotiation successful
0923 15:04:22.812116 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":99,"thread_start_us":71,"threads_started":1}
I20260923 15:04:22.812431 28890 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:22.792725 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.131:40593 (local address 127.13.75.129:60939)
0923 15:04:22.792784 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0923 15:04:22.792800 (+ 16us) client_negotiation.cc:174] Beginning negotiation
0923 15:04:22.792895 (+ 95us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:04:22.794023 (+ 1128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:04:22.794026 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:04:22.794546 (+ 520us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:04:22.794816 (+ 270us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:04:22.794824 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:04:22.796591 (+ 1767us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:04:22.796591 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:04:22.797944 (+ 1353us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:04:22.797947 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:04:22.798868 (+ 921us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:04:22.798870 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:04:22.799073 (+ 203us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:04:22.805915 (+ 6842us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:04:22.805932 (+ 17us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:04:22.808540 (+ 2608us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:04:22.810562 (+ 2022us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:04:22.810563 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:04:22.810563 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:04:22.810808 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:04:22.811127 (+ 319us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:04:22.811128 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:04:22.811128 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:04:22.811243 (+ 115us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:04:22.811680 (+ 437us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:04:22.811681 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:04:22.811804 (+ 123us) client_negotiation.cc:769] Sending connection context
0923 15:04:22.811917 (+ 113us) client_negotiation.cc:240] Negotiation successful
0923 15:04:22.812245 (+ 328us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":30,"mutex_wait_us":292}
W20260923 15:04:22.813814 28505 leader_election.cc:341] T 4db10a4e0b6a404f94257b46eb1ac368 P f6496d7f1e484285ba207e9b3f30edd8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5c449438d1ea448f8c0518eac20b8194 (127.13.75.130:36107): Not found: Tablet not found: 4db10a4e0b6a404f94257b46eb1ac368
W20260923 15:04:22.813913 28505 leader_election.cc:341] T 4db10a4e0b6a404f94257b46eb1ac368 P f6496d7f1e484285ba207e9b3f30edd8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 53f9c19a5c7449faa9b3897191c6cfc6 (127.13.75.131:40593): Not found: Tablet not found: 4db10a4e0b6a404f94257b46eb1ac368
I20260923 15:04:22.813953 28505 leader_election.cc:302] T 4db10a4e0b6a404f94257b46eb1ac368 P f6496d7f1e484285ba207e9b3f30edd8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f6496d7f1e484285ba207e9b3f30edd8; no voters: 53f9c19a5c7449faa9b3897191c6cfc6, 5c449438d1ea448f8c0518eac20b8194
I20260923 15:04:22.814059 28893 raft_consensus.cc:2726] T 4db10a4e0b6a404f94257b46eb1ac368 P f6496d7f1e484285ba207e9b3f30edd8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260923 15:04:23.481436 28897 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:23.481182 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:46259 (local address 127.13.75.129:48939)
0923 15:04:23.481339 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0923 15:04:23.481381 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.190:46259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":64,"threads_started":1}
I20260923 15:04:23.534312 28898 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:23.534082 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:46259 (local address 127.13.75.130:56585)
0923 15:04:23.534224 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0923 15:04:23.534262 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.190:46259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":67,"threads_started":1}
I20260923 15:04:23.662369 28899 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:23.662051 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:46259 (local address 127.13.75.131:41969)
0923 15:04:23.662240 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0923 15:04:23.662308 (+ 68us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.190:46259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":77,"threads_started":1}
I20260923 15:04:24.482271 28900 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:24.481993 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:46259 (local address 127.13.75.129:57433)
0923 15:04:24.482141 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0923 15:04:24.482177 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.190:46259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":87,"threads_started":1}
I20260923 15:04:24.535080 28901 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:24.534814 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:46259 (local address 127.13.75.130:49161)
0923 15:04:24.534992 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0923 15:04:24.535036 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.190:46259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":159,"thread_start_us":128,"threads_started":1}
I20260923 15:04:24.663132 28902 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:24.662861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:46259 (local address 127.13.75.131:42223)
0923 15:04:24.663057 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0923 15:04:24.663091 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.190:46259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":181,"thread_start_us":144,"threads_started":1}
I20260923 15:04:24.922948 28903 raft_consensus.cc:482] T 4db10a4e0b6a404f94257b46eb1ac368 P f6496d7f1e484285ba207e9b3f30edd8 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 15:04:24.923063 28903 raft_consensus.cc:504] T 4db10a4e0b6a404f94257b46eb1ac368 P f6496d7f1e484285ba207e9b3f30edd8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53f9c19a5c7449faa9b3897191c6cfc6" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 40593 } } peers { permanent_uuid: "f6496d7f1e484285ba207e9b3f30edd8" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 33885 } } peers { permanent_uuid: "5c449438d1ea448f8c0518eac20b8194" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 36107 } }
I20260923 15:04:24.923329 28903 leader_election.cc:288] T 4db10a4e0b6a404f94257b46eb1ac368 P f6496d7f1e484285ba207e9b3f30edd8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 53f9c19a5c7449faa9b3897191c6cfc6 (127.13.75.131:40593), 5c449438d1ea448f8c0518eac20b8194 (127.13.75.130:36107)
W20260923 15:04:24.923887 28505 leader_election.cc:341] T 4db10a4e0b6a404f94257b46eb1ac368 P f6496d7f1e484285ba207e9b3f30edd8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 53f9c19a5c7449faa9b3897191c6cfc6 (127.13.75.131:40593): Not found: Tablet not found: 4db10a4e0b6a404f94257b46eb1ac368
W20260923 15:04:24.923969 28505 leader_election.cc:341] T 4db10a4e0b6a404f94257b46eb1ac368 P f6496d7f1e484285ba207e9b3f30edd8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5c449438d1ea448f8c0518eac20b8194 (127.13.75.130:36107): Not found: Tablet not found: 4db10a4e0b6a404f94257b46eb1ac368
I20260923 15:04:24.923986 28505 leader_election.cc:302] T 4db10a4e0b6a404f94257b46eb1ac368 P f6496d7f1e484285ba207e9b3f30edd8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f6496d7f1e484285ba207e9b3f30edd8; no voters: 53f9c19a5c7449faa9b3897191c6cfc6, 5c449438d1ea448f8c0518eac20b8194
I20260923 15:04:24.924057 28903 raft_consensus.cc:2726] T 4db10a4e0b6a404f94257b46eb1ac368 P f6496d7f1e484285ba207e9b3f30edd8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260923 15:04:25.483067 28904 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:25.482796 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:46259 (local address 127.13.75.129:59479)
0923 15:04:25.482969 (+ 173us) negotiation.cc:107] Waiting for socket to connect
0923 15:04:25.483018 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.190:46259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":61,"threads_started":1}
I20260923 15:04:25.535848 28905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:25.535592 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:46259 (local address 127.13.75.130:46041)
0923 15:04:25.535753 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0923 15:04:25.535801 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.190:46259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":74,"threads_started":1}
I20260923 15:04:25.663887 28906 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:25.663636 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:46259 (local address 127.13.75.131:47219)
0923 15:04:25.663779 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0923 15:04:25.663829 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.190:46259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":87,"threads_started":1}
I20260923 15:04:26.483891 28907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:26.483646 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:46259 (local address 127.13.75.129:57909)
0923 15:04:26.483794 (+ 148us) negotiation.cc:107] Waiting for socket to connect
0923 15:04:26.483843 (+ 49us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.190:46259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":73,"threads_started":1}
I20260923 15:04:26.536690 28908 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:26.536432 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:46259 (local address 127.13.75.130:49191)
0923 15:04:26.536600 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0923 15:04:26.536645 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.190:46259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":73,"threads_started":1}
I20260923 15:04:26.664780 28909 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:26.664509 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:46259 (local address 127.13.75.131:51269)
0923 15:04:26.664686 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0923 15:04:26.664733 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.190:46259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":155,"thread_start_us":86,"threads_started":1}
I20260923 15:04:26.873427 28910 raft_consensus.cc:482] T 4db10a4e0b6a404f94257b46eb1ac368 P f6496d7f1e484285ba207e9b3f30edd8 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 15:04:26.873533 28910 raft_consensus.cc:504] T 4db10a4e0b6a404f94257b46eb1ac368 P f6496d7f1e484285ba207e9b3f30edd8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53f9c19a5c7449faa9b3897191c6cfc6" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 40593 } } peers { permanent_uuid: "f6496d7f1e484285ba207e9b3f30edd8" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 33885 } } peers { permanent_uuid: "5c449438d1ea448f8c0518eac20b8194" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 36107 } }
I20260923 15:04:26.873724 28910 leader_election.cc:288] T 4db10a4e0b6a404f94257b46eb1ac368 P f6496d7f1e484285ba207e9b3f30edd8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 53f9c19a5c7449faa9b3897191c6cfc6 (127.13.75.131:40593), 5c449438d1ea448f8c0518eac20b8194 (127.13.75.130:36107)
W20260923 15:04:26.874241 28505 leader_election.cc:341] T 4db10a4e0b6a404f94257b46eb1ac368 P f6496d7f1e484285ba207e9b3f30edd8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5c449438d1ea448f8c0518eac20b8194 (127.13.75.130:36107): Not found: Tablet not found: 4db10a4e0b6a404f94257b46eb1ac368
W20260923 15:04:26.874327 28505 leader_election.cc:341] T 4db10a4e0b6a404f94257b46eb1ac368 P f6496d7f1e484285ba207e9b3f30edd8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 53f9c19a5c7449faa9b3897191c6cfc6 (127.13.75.131:40593): Not found: Tablet not found: 4db10a4e0b6a404f94257b46eb1ac368
I20260923 15:04:26.874378 28505 leader_election.cc:302] T 4db10a4e0b6a404f94257b46eb1ac368 P f6496d7f1e484285ba207e9b3f30edd8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f6496d7f1e484285ba207e9b3f30edd8; no voters: 53f9c19a5c7449faa9b3897191c6cfc6, 5c449438d1ea448f8c0518eac20b8194
I20260923 15:04:26.874476 28910 raft_consensus.cc:2726] T 4db10a4e0b6a404f94257b46eb1ac368 P f6496d7f1e484285ba207e9b3f30edd8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260923 15:04:27.484771 28911 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:27.484517 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:46259 (local address 127.13.75.129:33269)
0923 15:04:27.484672 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0923 15:04:27.484717 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.190:46259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":79,"threads_started":1}
I20260923 15:04:27.537627 28912 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:27.537346 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:46259 (local address 127.13.75.130:53875)
0923 15:04:27.537517 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0923 15:04:27.537581 (+ 64us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.190:46259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":83,"threads_started":1}
I20260923 15:04:27.665571 28913 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:27.665300 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:46259 (local address 127.13.75.131:37675)
0923 15:04:27.665455 (+ 155us) negotiation.cc:107] Waiting for socket to connect
0923 15:04:27.665502 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.190:46259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":86,"threads_started":1}
I20260923 15:04:28.485559 28914 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:28.485324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:46259 (local address 127.13.75.129:38769)
0923 15:04:28.485468 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0923 15:04:28.485511 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.190:46259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":61,"threads_started":1}
I20260923 15:04:28.538307 28915 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:28.538090 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:46259 (local address 127.13.75.130:36135)
0923 15:04:28.538223 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0923 15:04:28.538261 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.190:46259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":64,"threads_started":1}
I20260923 15:04:28.666349 28916 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:28.666124 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:46259 (local address 127.13.75.131:36621)
0923 15:04:28.666261 (+ 137us) negotiation.cc:107] Waiting for socket to connect
0923 15:04:28.666301 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.190:46259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":66,"threads_started":1}
I20260923 15:04:29.486364 28917 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:29.486066 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:46259 (local address 127.13.75.129:46693)
0923 15:04:29.486225 (+ 159us) negotiation.cc:107] Waiting for socket to connect
0923 15:04:29.486312 (+ 87us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.190:46259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":77,"threads_started":1}
I20260923 15:04:29.539270 28918 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:29.539004 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:46259 (local address 127.13.75.130:40673)
0923 15:04:29.539167 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0923 15:04:29.539214 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.190:46259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":78,"threads_started":1}
I20260923 15:04:29.667176 28919 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:29.666956 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:46259 (local address 127.13.75.131:34165)
0923 15:04:29.667087 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0923 15:04:29.667128 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.190:46259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":58,"threads_started":1}
I20260923 15:04:30.487291 28920 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:30.487001 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:46259 (local address 127.13.75.129:37299)
0923 15:04:30.487199 (+ 198us) negotiation.cc:107] Waiting for socket to connect
0923 15:04:30.487244 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.190:46259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":176,"thread_start_us":84,"threads_started":1}
I20260923 15:04:30.540223 28921 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:30.539950 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:46259 (local address 127.13.75.130:39677)
0923 15:04:30.540128 (+ 178us) negotiation.cc:107] Waiting for socket to connect
0923 15:04:30.540173 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.190:46259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":151,"thread_start_us":70,"threads_started":1}
I20260923 15:04:30.668052 28922 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:30.667770 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:46259 (local address 127.13.75.131:58749)
0923 15:04:30.667939 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0923 15:04:30.668004 (+ 65us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.190:46259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":83,"threads_started":1}
I20260923 15:04:31.259830 28923 raft_consensus.cc:482] T 4db10a4e0b6a404f94257b46eb1ac368 P f6496d7f1e484285ba207e9b3f30edd8 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 15:04:31.259935 28923 raft_consensus.cc:504] T 4db10a4e0b6a404f94257b46eb1ac368 P f6496d7f1e484285ba207e9b3f30edd8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53f9c19a5c7449faa9b3897191c6cfc6" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 40593 } } peers { permanent_uuid: "f6496d7f1e484285ba207e9b3f30edd8" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 33885 } } peers { permanent_uuid: "5c449438d1ea448f8c0518eac20b8194" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 36107 } }
I20260923 15:04:31.260155 28923 leader_election.cc:288] T 4db10a4e0b6a404f94257b46eb1ac368 P f6496d7f1e484285ba207e9b3f30edd8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 53f9c19a5c7449faa9b3897191c6cfc6 (127.13.75.131:40593), 5c449438d1ea448f8c0518eac20b8194 (127.13.75.130:36107)
W20260923 15:04:31.260648 28505 leader_election.cc:341] T 4db10a4e0b6a404f94257b46eb1ac368 P f6496d7f1e484285ba207e9b3f30edd8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 53f9c19a5c7449faa9b3897191c6cfc6 (127.13.75.131:40593): Not found: Tablet not found: 4db10a4e0b6a404f94257b46eb1ac368
W20260923 15:04:31.260737 28505 leader_election.cc:341] T 4db10a4e0b6a404f94257b46eb1ac368 P f6496d7f1e484285ba207e9b3f30edd8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5c449438d1ea448f8c0518eac20b8194 (127.13.75.130:36107): Not found: Tablet not found: 4db10a4e0b6a404f94257b46eb1ac368
I20260923 15:04:31.260799 28505 leader_election.cc:302] T 4db10a4e0b6a404f94257b46eb1ac368 P f6496d7f1e484285ba207e9b3f30edd8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f6496d7f1e484285ba207e9b3f30edd8; no voters: 53f9c19a5c7449faa9b3897191c6cfc6, 5c449438d1ea448f8c0518eac20b8194
I20260923 15:04:31.260876 28923 raft_consensus.cc:2726] T 4db10a4e0b6a404f94257b46eb1ac368 P f6496d7f1e484285ba207e9b3f30edd8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260923 15:04:31.488343 28924 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:31.487989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:46259 (local address 127.13.75.129:33027)
0923 15:04:31.488194 (+ 205us) negotiation.cc:107] Waiting for socket to connect
0923 15:04:31.488291 (+ 97us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.190:46259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":183,"thread_start_us":91,"threads_started":1}
I20260923 15:04:31.541160 28925 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:31.540844 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:46259 (local address 127.13.75.130:51953)
0923 15:04:31.541034 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0923 15:04:31.541089 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.190:46259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":86,"threads_started":1}
I20260923 15:04:31.669061 28926 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:31.668815 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:46259 (local address 127.13.75.131:50525)
0923 15:04:31.668948 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0923 15:04:31.668994 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.190:46259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":111,"thread_start_us":79,"threads_started":1}
I20260923 15:04:32.489287 28927 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:32.488996 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:46259 (local address 127.13.75.129:36767)
0923 15:04:32.489145 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0923 15:04:32.489187 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.190:46259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":65,"threads_started":1}
I20260923 15:04:32.542053 28928 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:32.541766 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:46259 (local address 127.13.75.130:44923)
0923 15:04:32.541917 (+ 151us) negotiation.cc:107] Waiting for socket to connect
0923 15:04:32.541971 (+ 54us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.190:46259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":72,"threads_started":1}
I20260923 15:04:32.669878 28929 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:32.669616 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:46259 (local address 127.13.75.131:52199)
0923 15:04:32.669786 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0923 15:04:32.669831 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.190:46259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":80,"threads_started":1}
I20260923 15:04:33.490139 28930 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:33.489894 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:46259 (local address 127.13.75.129:56157)
0923 15:04:33.490043 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0923 15:04:33.490088 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.190:46259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":71,"threads_started":1}
I20260923 15:04:33.542877 28931 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:33.542589 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:46259 (local address 127.13.75.130:44111)
0923 15:04:33.542782 (+ 193us) negotiation.cc:107] Waiting for socket to connect
0923 15:04:33.542828 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.190:46259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":69,"threads_started":1}
I20260923 15:04:33.670727 28932 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:33.670451 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:46259 (local address 127.13.75.131:35001)
0923 15:04:33.670638 (+ 187us) negotiation.cc:107] Waiting for socket to connect
0923 15:04:33.670680 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.190:46259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":163,"thread_start_us":83,"threads_started":1}
I20260923 15:04:34.491017 28933 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:34.490731 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:46259 (local address 127.13.75.129:59983)
0923 15:04:34.490861 (+ 130us) negotiation.cc:107] Waiting for socket to connect
0923 15:04:34.490970 (+ 109us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.190:46259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":113,"thread_start_us":61,"threads_started":1}
I20260923 15:04:34.501559 28934 raft_consensus.cc:482] T 4db10a4e0b6a404f94257b46eb1ac368 P f6496d7f1e484285ba207e9b3f30edd8 [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 15:04:34.501654 28934 raft_consensus.cc:504] T 4db10a4e0b6a404f94257b46eb1ac368 P f6496d7f1e484285ba207e9b3f30edd8 [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "53f9c19a5c7449faa9b3897191c6cfc6" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 40593 } } peers { permanent_uuid: "f6496d7f1e484285ba207e9b3f30edd8" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 33885 } } peers { permanent_uuid: "5c449438d1ea448f8c0518eac20b8194" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 36107 } }
I20260923 15:04:34.501792 28934 leader_election.cc:288] T 4db10a4e0b6a404f94257b46eb1ac368 P f6496d7f1e484285ba207e9b3f30edd8 [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers 53f9c19a5c7449faa9b3897191c6cfc6 (127.13.75.131:40593), 5c449438d1ea448f8c0518eac20b8194 (127.13.75.130:36107)
W20260923 15:04:34.502355 28505 leader_election.cc:341] T 4db10a4e0b6a404f94257b46eb1ac368 P f6496d7f1e484285ba207e9b3f30edd8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 53f9c19a5c7449faa9b3897191c6cfc6 (127.13.75.131:40593): Not found: Tablet not found: 4db10a4e0b6a404f94257b46eb1ac368
W20260923 15:04:34.502434 28505 leader_election.cc:341] T 4db10a4e0b6a404f94257b46eb1ac368 P f6496d7f1e484285ba207e9b3f30edd8 [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 5c449438d1ea448f8c0518eac20b8194 (127.13.75.130:36107): Not found: Tablet not found: 4db10a4e0b6a404f94257b46eb1ac368
I20260923 15:04:34.502475 28505 leader_election.cc:302] T 4db10a4e0b6a404f94257b46eb1ac368 P f6496d7f1e484285ba207e9b3f30edd8 [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: f6496d7f1e484285ba207e9b3f30edd8; no voters: 53f9c19a5c7449faa9b3897191c6cfc6, 5c449438d1ea448f8c0518eac20b8194
I20260923 15:04:34.502600 28934 raft_consensus.cc:2726] T 4db10a4e0b6a404f94257b46eb1ac368 P f6496d7f1e484285ba207e9b3f30edd8 [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260923 15:04:34.543902 28935 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:34.543605 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:46259 (local address 127.13.75.130:48193)
0923 15:04:34.543789 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0923 15:04:34.543834 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.190:46259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":164,"thread_start_us":88,"threads_started":1}
I20260923 15:04:34.671665 28936 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:34.671390 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:46259 (local address 127.13.75.131:32865)
0923 15:04:34.671567 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0923 15:04:34.671609 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.190:46259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":158,"thread_start_us":91,"threads_started":1}
I20260923 15:04:35.491722 28937 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:35.491519 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:46259 (local address 127.13.75.129:39155)
0923 15:04:35.491641 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0923 15:04:35.491670 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.190:46259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":99,"thread_start_us":56,"threads_started":1}
I20260923 15:04:35.544641 28938 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:35.544394 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:46259 (local address 127.13.75.130:58291)
0923 15:04:35.544551 (+ 157us) negotiation.cc:107] Waiting for socket to connect
0923 15:04:35.544591 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.190:46259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":71,"threads_started":1}
I20260923 15:04:35.672360 28939 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:35.672131 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:46259 (local address 127.13.75.131:57087)
0923 15:04:35.672276 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0923 15:04:35.672313 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.190:46259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":85,"threads_started":1}
I20260923 15:04:36.492681 28940 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:36.492425 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:46259 (local address 127.13.75.129:52553)
0923 15:04:36.492587 (+ 162us) negotiation.cc:107] Waiting for socket to connect
0923 15:04:36.492633 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.190:46259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":70,"threads_started":1}
I20260923 15:04:36.545468 28941 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:36.545246 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:46259 (local address 127.13.75.130:50569)
0923 15:04:36.545379 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0923 15:04:36.545422 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.190:46259: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":62,"threads_started":1}
I20260923 15:04:36.662734 13614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKC4soy/build/release/bin/kudu with pid 28460
I20260923 15:04:36.667604 13614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKC4soy/build/release/bin/kudu with pid 28624
I20260923 15:04:36.671962 13614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKC4soy/build/release/bin/kudu with pid 28757
2026-09-23T15:04:36Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedIPKICert (18164 ms)
[ RUN ] SecurityITest.TestEnableAndDisableEncryptedTSK
Loading random data
Initializing database '/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 15:04:36 dist-test-slave-cnb0 krb5kdc[28944](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 15:04:36 dist-test-slave-cnb0 krb5kdc[28944](info): set up 2 sockets
Sep 23 15:04:36 dist-test-slave-cnb0 krb5kdc[28944](info): commencing operation
krb5kdc: starting...
W20260923 15:04:37.728119 13614 mini_kdc.cc:121] Time spent starting KDC: real 1.036s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/krb5kdc/test-user.keytab.
Sep 23 15:04:37 dist-test-slave-cnb0 krb5kdc[28944](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175877, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T15:04:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T15:04:37Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.75.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.75.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.75.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.190.keytab.
Entry for principal kudu/127.13.75.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.190.keytab.
I20260923 15:04:37.853477 13614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKC4soy/build/release/bin/kudu
/tmp/dist-test-taskKC4soy/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.75.190:40941
--webserver_interface=127.13.75.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.190.keytab
--principal=kudu/127.13.75.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.75.148:37659
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.75.190:40941
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:04:37.935340 28957 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:04:37.935550 28957 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:04:37.935573 28957 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:04:37.936753 28957 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260923 15:04:37.936793 28957 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260923 15:04:37.936806 28957 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260923 15:04:37.936818 28957 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 15:04:37.936831 28957 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:04:37.936844 28957 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:04:37.936893 28957 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 15:04:37.936910 28957 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 15:04:37.938480 28957 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.75.148:37659
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.13.75.190:40941
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.190.keytab
--principal=kudu/127.13.75.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.75.190:40941
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.75.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.28957
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:04:37.938732 28957 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:04:37.938993 28957 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:04:37.944537 28965 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:04:37.944495 28963 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:04:37.944468 28957 server_base.cc:1034] running on GCE node
W20260923 15:04:37.944483 28962 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:04:37.945003 28957 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:04:37.945266 28957 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:04:37.946437 28957 hybrid_clock.cc:648] HybridClock initialized: now 1790175877946426 us; error 30 us; skew 500 ppm
Sep 23 15:04:37 dist-test-slave-cnb0 krb5kdc[28944](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175877, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:04:37.948972 28957 init.cc:377] Logged in from keytab as kudu/127.13.75.190@KRBTEST.COM (short username kudu)
I20260923 15:04:37.949458 28957 webserver.cc:460] Webserver started at http://127.13.75.190:35917/ using document root <none> and password file <none>
I20260923 15:04:37.949697 28957 fs_manager.cc:359] Metadata directory not provided
I20260923 15:04:37.949767 28957 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:04:37.949904 28957 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:04:37.951066 28957 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/minicluster-data/master-0/data/instance:
uuid: "842c3ea43bfc4cda9118b7f7d8dd212c"
format_stamp: "Formatted at 2026-09-23 15:04:37 on dist-test-slave-cnb0"
server_key: "c5bd9a1448bd7566ed2d489c8e7943bb"
server_key_iv: "59b41eac661e9094ff8487caada62cc3"
server_key_version: "encryptionkey@0"
I20260923 15:04:37.951419 28957 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/minicluster-data/master-0/wal/instance:
uuid: "842c3ea43bfc4cda9118b7f7d8dd212c"
format_stamp: "Formatted at 2026-09-23 15:04:37 on dist-test-slave-cnb0"
server_key: "c5bd9a1448bd7566ed2d489c8e7943bb"
server_key_iv: "59b41eac661e9094ff8487caada62cc3"
server_key_version: "encryptionkey@0"
I20260923 15:04:37.952762 28957 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260923 15:04:37.953652 28972 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:04:37.953951 28957 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260923 15:04:37.954059 28957 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/minicluster-data/master-0/data,/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/minicluster-data/master-0/wal
uuid: "842c3ea43bfc4cda9118b7f7d8dd212c"
format_stamp: "Formatted at 2026-09-23 15:04:37 on dist-test-slave-cnb0"
server_key: "c5bd9a1448bd7566ed2d489c8e7943bb"
server_key_iv: "59b41eac661e9094ff8487caada62cc3"
server_key_version: "encryptionkey@0"
I20260923 15:04:37.954136 28957 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:04:37.963769 28957 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:04:37.965734 28957 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:04:37.965912 28957 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:04:37.970476 28957 rpc_server.cc:307] RPC server started. Bound to: 127.13.75.190:40941
I20260923 15:04:37.970525 29024 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.75.190:40941 every 8 connection(s)
I20260923 15:04:37.970959 28957 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/minicluster-data/master-0/data/info.pb
I20260923 15:04:37.971880 29025 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:04:37.974135 29025 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 842c3ea43bfc4cda9118b7f7d8dd212c: Bootstrap starting.
I20260923 15:04:37.974717 29025 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 842c3ea43bfc4cda9118b7f7d8dd212c: Neither blocks nor log segments found. Creating new log.
I20260923 15:04:37.975064 29025 log.cc:824] T 00000000000000000000000000000000 P 842c3ea43bfc4cda9118b7f7d8dd212c: Log is configured to *not* fsync() on all Append() calls
I20260923 15:04:37.975733 29025 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 842c3ea43bfc4cda9118b7f7d8dd212c: No bootstrap required, opened a new log
I20260923 15:04:37.977341 29025 raft_consensus.cc:348] T 00000000000000000000000000000000 P 842c3ea43bfc4cda9118b7f7d8dd212c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "842c3ea43bfc4cda9118b7f7d8dd212c" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 40941 } }
I20260923 15:04:37.977479 29025 raft_consensus.cc:374] T 00000000000000000000000000000000 P 842c3ea43bfc4cda9118b7f7d8dd212c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:04:37.977501 29025 raft_consensus.cc:729] T 00000000000000000000000000000000 P 842c3ea43bfc4cda9118b7f7d8dd212c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 842c3ea43bfc4cda9118b7f7d8dd212c, State: Initialized, Role: FOLLOWER
I20260923 15:04:37.977648 29025 consensus_queue.cc:260] T 00000000000000000000000000000000 P 842c3ea43bfc4cda9118b7f7d8dd212c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "842c3ea43bfc4cda9118b7f7d8dd212c" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 40941 } }
I20260923 15:04:37.977721 29025 raft_consensus.cc:388] T 00000000000000000000000000000000 P 842c3ea43bfc4cda9118b7f7d8dd212c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 15:04:37.977743 29025 raft_consensus.cc:482] T 00000000000000000000000000000000 P 842c3ea43bfc4cda9118b7f7d8dd212c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 15:04:37.977839 29025 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 842c3ea43bfc4cda9118b7f7d8dd212c [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:04:37.978401 29025 raft_consensus.cc:504] T 00000000000000000000000000000000 P 842c3ea43bfc4cda9118b7f7d8dd212c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "842c3ea43bfc4cda9118b7f7d8dd212c" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 40941 } }
I20260923 15:04:37.978554 29025 leader_election.cc:302] T 00000000000000000000000000000000 P 842c3ea43bfc4cda9118b7f7d8dd212c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 842c3ea43bfc4cda9118b7f7d8dd212c; no voters:
I20260923 15:04:37.978614 13614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKC4soy/build/release/bin/kudu as pid 28957
I20260923 15:04:37.978703 13614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/minicluster-data/master-0/wal/instance
I20260923 15:04:37.978731 29025 leader_election.cc:288] T 00000000000000000000000000000000 P 842c3ea43bfc4cda9118b7f7d8dd212c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 15:04:37.978777 29028 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 842c3ea43bfc4cda9118b7f7d8dd212c [term 1 FOLLOWER]: Leader election won for term 1
I20260923 15:04:37.978943 13614 external_mini_cluster.cc:1428] Setting key ef97b03e62975f4cc70762b6a4536991
I20260923 15:04:37.978992 29028 raft_consensus.cc:686] T 00000000000000000000000000000000 P 842c3ea43bfc4cda9118b7f7d8dd212c [term 1 LEADER]: Becoming Leader. State: Replica: 842c3ea43bfc4cda9118b7f7d8dd212c, State: Running, Role: LEADER
I20260923 15:04:37.979034 29025 sys_catalog.cc:564] T 00000000000000000000000000000000 P 842c3ea43bfc4cda9118b7f7d8dd212c [sys.catalog]: configured and running, proceeding with master startup.
I20260923 15:04:37.979075 29028 consensus_queue.cc:237] T 00000000000000000000000000000000 P 842c3ea43bfc4cda9118b7f7d8dd212c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "842c3ea43bfc4cda9118b7f7d8dd212c" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 40941 } }
I20260923 15:04:37.979889 29029 sys_catalog.cc:455] T 00000000000000000000000000000000 P 842c3ea43bfc4cda9118b7f7d8dd212c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 842c3ea43bfc4cda9118b7f7d8dd212c. Latest consensus state: current_term: 1 leader_uuid: "842c3ea43bfc4cda9118b7f7d8dd212c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "842c3ea43bfc4cda9118b7f7d8dd212c" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 40941 } } }
I20260923 15:04:37.979876 29030 sys_catalog.cc:455] T 00000000000000000000000000000000 P 842c3ea43bfc4cda9118b7f7d8dd212c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "842c3ea43bfc4cda9118b7f7d8dd212c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "842c3ea43bfc4cda9118b7f7d8dd212c" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 40941 } } }
I20260923 15:04:37.979990 29029 sys_catalog.cc:458] T 00000000000000000000000000000000 P 842c3ea43bfc4cda9118b7f7d8dd212c [sys.catalog]: This master's current role is: LEADER
I20260923 15:04:37.979995 29030 sys_catalog.cc:458] T 00000000000000000000000000000000 P 842c3ea43bfc4cda9118b7f7d8dd212c [sys.catalog]: This master's current role is: LEADER
I20260923 15:04:37.980199 29041 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 15:04:37.981479 29041 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 15:04:37.984062 29041 catalog_manager.cc:1347] Generated new cluster ID: 5e9ad76930f74565ae7880afecf8f882
I20260923 15:04:37.984118 29041 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 23 15:04:37 dist-test-slave-cnb0 krb5kdc[28944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175877, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.75.190@KRBTEST.COM
I20260923 15:04:37.992345 29034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:37.979499 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48766 (local address 127.13.75.190:40941)
0923 15:04:37.979883 (+ 384us) server_negotiation.cc:206] Beginning negotiation
0923 15:04:37.979892 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0923 15:04:37.979927 (+ 35us) server_negotiation.cc:407] Connection header received
0923 15:04:37.980058 (+ 131us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:04:37.980066 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:04:37.980145 (+ 79us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:04:37.980327 (+ 182us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:04:37.981095 (+ 768us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:04:37.982135 (+ 1040us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:04:37.983211 (+ 1076us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:04:37.983831 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:04:37.983945 (+ 114us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:04:37.989099 (+ 5154us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:04:37.989104 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:04:37.989104 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:04:37.989121 (+ 17us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:04:37.990726 (+ 1605us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:04:37.991144 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:04:37.991147 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:04:37.991147 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:04:37.991242 (+ 95us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:04:37.991449 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:04:37.991450 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:04:37.991450 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:04:37.991745 (+ 295us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:04:37.991908 (+ 163us) server_negotiation.cc:1035] Waiting for connection context
0923 15:04:37.991985 (+ 77us) server_negotiation.cc:299] Negotiation successful
0923 15:04:37.992127 (+ 142us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":334,"thread_start_us":62,"threads_started":1}
I20260923 15:04:37.997578 29041 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 15:04:37.998142 29041 catalog_manager.cc:1504] Loading token signing keys...
I20260923 15:04:38.002588 29041 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 842c3ea43bfc4cda9118b7f7d8dd212c: Generated new TSK 0
I20260923 15:04:38.002796 29041 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.75.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.75.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.75.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.129.keytab.
Entry for principal kudu/127.13.75.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.129.keytab.
I20260923 15:04:38.035535 13614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKC4soy/build/release/bin/kudu
/tmp/dist-test-taskKC4soy/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.75.129:0
--local_ip_for_outbound_sockets=127.13.75.129
--webserver_interface=127.13.75.129
--webserver_port=0
--tserver_master_addrs=127.13.75.190:40941
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.129.keytab
--principal=kudu/127.13.75.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.75.148:37659
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:04:38.118873 29051 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:04:38.119091 29051 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:04:38.119115 29051 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:04:38.120317 29051 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:04:38.120360 29051 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:04:38.120388 29051 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.75.129
I20260923 15:04:38.121845 29051 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.75.148:37659
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.129.keytab
--principal=kudu/127.13.75.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.75.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.75.129
--webserver_port=0
--tserver_master_addrs=127.13.75.190:40941
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29051
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.75.129
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:04:38.122035 29051 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:04:38.122218 29051 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:04:38.127228 29057 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:04:38.127215 29059 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:04:38.127215 29056 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:04:38.127501 29051 server_base.cc:1034] running on GCE node
I20260923 15:04:38.127732 29051 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:04:38.127986 29051 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:04:38.129169 29051 hybrid_clock.cc:648] HybridClock initialized: now 1790175878129147 us; error 39 us; skew 500 ppm
Sep 23 15:04:38 dist-test-slave-cnb0 krb5kdc[28944](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175878, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:04:38.132619 29051 init.cc:377] Logged in from keytab as kudu/127.13.75.129@KRBTEST.COM (short username kudu)
I20260923 15:04:38.133297 29051 webserver.cc:460] Webserver started at http://127.13.75.129:41567/ using document root <none> and password file <none>
I20260923 15:04:38.133662 29051 fs_manager.cc:359] Metadata directory not provided
I20260923 15:04:38.133805 29051 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:04:38.133981 29051 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:04:38.135344 29051 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/minicluster-data/ts-0/data/instance:
uuid: "125b34b86b954465bc1945b9733a2b0a"
format_stamp: "Formatted at 2026-09-23 15:04:38 on dist-test-slave-cnb0"
server_key: "30fb4b9008d66fe51a6d9aab609f2492"
server_key_iv: "41a52838ed269bdd2774e65fb947e496"
server_key_version: "encryptionkey@0"
I20260923 15:04:38.135725 29051 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/minicluster-data/ts-0/wal/instance:
uuid: "125b34b86b954465bc1945b9733a2b0a"
format_stamp: "Formatted at 2026-09-23 15:04:38 on dist-test-slave-cnb0"
server_key: "30fb4b9008d66fe51a6d9aab609f2492"
server_key_iv: "41a52838ed269bdd2774e65fb947e496"
server_key_version: "encryptionkey@0"
I20260923 15:04:38.137231 29051 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 15:04:38.138175 29066 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:04:38.138345 29051 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 15:04:38.138454 29051 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/minicluster-data/ts-0/wal
uuid: "125b34b86b954465bc1945b9733a2b0a"
format_stamp: "Formatted at 2026-09-23 15:04:38 on dist-test-slave-cnb0"
server_key: "30fb4b9008d66fe51a6d9aab609f2492"
server_key_iv: "41a52838ed269bdd2774e65fb947e496"
server_key_version: "encryptionkey@0"
I20260923 15:04:38.138536 29051 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:04:38.178761 29051 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:04:38.181452 29051 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:04:38.181620 29051 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:04:38.181890 29051 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 15:04:38.182216 29051 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 15:04:38.182252 29051 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 15:04:38.182276 29051 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 15:04:38.182355 29051 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 15:04:38.188043 29051 rpc_server.cc:307] RPC server started. Bound to: 127.13.75.129:39471
I20260923 15:04:38.188087 29179 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.75.129:39471 every 8 connection(s)
I20260923 15:04:38.188457 29051 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/minicluster-data/ts-0/data/info.pb
I20260923 15:04:38.191454 13614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKC4soy/build/release/bin/kudu as pid 29051
I20260923 15:04:38.191568 13614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/minicluster-data/ts-0/wal/instance
I20260923 15:04:38.191761 13614 external_mini_cluster.cc:1428] Setting key 1ad161ba22fc45cf3047b0814ab50eb8
Sep 23 15:04:38 dist-test-slave-cnb0 krb5kdc[28944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175878, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.129@KRBTEST.COM for kudu/127.13.75.190@KRBTEST.COM
I20260923 15:04:38.199498 29034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:38.189194 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.129:48023 (local address 127.13.75.190:40941)
0923 15:04:38.189311 (+ 117us) server_negotiation.cc:206] Beginning negotiation
0923 15:04:38.189313 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:04:38.189479 (+ 166us) server_negotiation.cc:407] Connection header received
0923 15:04:38.189995 (+ 516us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:04:38.189997 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:04:38.190011 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:04:38.190076 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:04:38.191147 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:04:38.191728 (+ 581us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:04:38.193636 (+ 1908us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:04:38.193911 (+ 275us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:04:38.194004 (+ 93us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:04:38.196359 (+ 2355us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:04:38.196361 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:04:38.196361 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:04:38.196373 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:04:38.197960 (+ 1587us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:04:38.198423 (+ 463us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:04:38.198424 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:04:38.198425 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:04:38.198488 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:04:38.198756 (+ 268us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:04:38.198757 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:04:38.198758 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:04:38.198932 (+ 174us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:04:38.199001 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0923 15:04:38.199298 (+ 297us) server_negotiation.cc:299] Negotiation successful
0923 15:04:38.199397 (+ 99us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":88}
I20260923 15:04:38.199712 29182 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:38.189172 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:40941 (local address 127.13.75.129:48023)
0923 15:04:38.189358 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0923 15:04:38.189408 (+ 50us) client_negotiation.cc:174] Beginning negotiation
0923 15:04:38.189878 (+ 470us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:04:38.190171 (+ 293us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:04:38.190174 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:04:38.190782 (+ 608us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:04:38.191053 (+ 271us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:04:38.191058 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:04:38.191879 (+ 821us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:04:38.191880 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:04:38.193518 (+ 1638us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:04:38.193521 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:04:38.194012 (+ 491us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:04:38.194013 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:04:38.194158 (+ 145us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:04:38.194732 (+ 574us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:04:38.194740 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:04:38.196234 (+ 1494us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:04:38.198068 (+ 1834us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:04:38.198076 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:04:38.198079 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:04:38.198329 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:04:38.198559 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:04:38.198560 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:04:38.198561 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:04:38.198678 (+ 117us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:04:38.198991 (+ 313us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:04:38.198999 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:04:38.199150 (+ 151us) client_negotiation.cc:769] Sending connection context
0923 15:04:38.199268 (+ 118us) client_negotiation.cc:240] Negotiation successful
0923 15:04:38.199448 (+ 180us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":106,"threads_started":1}
WARNING: no policy specified for kudu/127.13.75.130@KRBTEST.COM; defaulting to no policy
I20260923 15:04:38.200304 29180 heartbeater.cc:347] Connected to a master server at 127.13.75.190:40941
I20260923 15:04:38.200431 29180 heartbeater.cc:464] Registering TS with master...
I20260923 15:04:38.200625 29180 heartbeater.cc:511] Master 127.13.75.190:40941 requested a full tablet report, sending...
I20260923 15:04:38.201124 28989 ts_manager.cc:194] Registered new tserver with Master: 125b34b86b954465bc1945b9733a2b0a (127.13.75.129:39471)
I20260923 15:04:38.201828 28989 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.75.129@KRBTEST.COM'} at 127.13.75.129:48023
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.75.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.75.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.130.keytab.
Entry for principal kudu/127.13.75.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.130.keytab.
I20260923 15:04:38.219190 13614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKC4soy/build/release/bin/kudu
/tmp/dist-test-taskKC4soy/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.75.130:0
--local_ip_for_outbound_sockets=127.13.75.130
--webserver_interface=127.13.75.130
--webserver_port=0
--tserver_master_addrs=127.13.75.190:40941
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.130.keytab
--principal=kudu/127.13.75.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.75.148:37659
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:04:38.300348 29185 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:04:38.300550 29185 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:04:38.300573 29185 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:04:38.301689 29185 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:04:38.301729 29185 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:04:38.301755 29185 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.75.130
I20260923 15:04:38.303330 29185 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.75.148:37659
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.130.keytab
--principal=kudu/127.13.75.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.75.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.75.130
--webserver_port=0
--tserver_master_addrs=127.13.75.190:40941
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29185
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.75.130
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:04:38.303529 29185 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:04:38.303710 29185 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:04:38.308955 29193 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:04:38.309115 29185 server_base.cc:1034] running on GCE node
W20260923 15:04:38.308995 29191 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:04:38.308959 29190 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:04:38.309463 29185 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:04:38.309721 29185 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:04:38.310868 29185 hybrid_clock.cc:648] HybridClock initialized: now 1790175878310852 us; error 29 us; skew 500 ppm
Sep 23 15:04:38 dist-test-slave-cnb0 krb5kdc[28944](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175878, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:04:38.313637 29185 init.cc:377] Logged in from keytab as kudu/127.13.75.130@KRBTEST.COM (short username kudu)
I20260923 15:04:38.314121 29185 webserver.cc:460] Webserver started at http://127.13.75.130:46595/ using document root <none> and password file <none>
I20260923 15:04:38.314380 29185 fs_manager.cc:359] Metadata directory not provided
I20260923 15:04:38.314432 29185 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:04:38.314534 29185 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:04:38.315641 29185 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/minicluster-data/ts-1/data/instance:
uuid: "fd45cc8453994d0fabe2a44f19a74598"
format_stamp: "Formatted at 2026-09-23 15:04:38 on dist-test-slave-cnb0"
server_key: "b82ae8a1cf14878afae4c201907dbf64"
server_key_iv: "2e7ee59bb01f910981ccc6d03c7cb38e"
server_key_version: "encryptionkey@0"
I20260923 15:04:38.315948 29185 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/minicluster-data/ts-1/wal/instance:
uuid: "fd45cc8453994d0fabe2a44f19a74598"
format_stamp: "Formatted at 2026-09-23 15:04:38 on dist-test-slave-cnb0"
server_key: "b82ae8a1cf14878afae4c201907dbf64"
server_key_iv: "2e7ee59bb01f910981ccc6d03c7cb38e"
server_key_version: "encryptionkey@0"
I20260923 15:04:38.317343 29185 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260923 15:04:38.318131 29200 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:04:38.318378 29185 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:04:38.318490 29185 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/minicluster-data/ts-1/wal
uuid: "fd45cc8453994d0fabe2a44f19a74598"
format_stamp: "Formatted at 2026-09-23 15:04:38 on dist-test-slave-cnb0"
server_key: "b82ae8a1cf14878afae4c201907dbf64"
server_key_iv: "2e7ee59bb01f910981ccc6d03c7cb38e"
server_key_version: "encryptionkey@0"
I20260923 15:04:38.318567 29185 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:04:38.325590 29185 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:04:38.327898 29185 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:04:38.328071 29185 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:04:38.328357 29185 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 15:04:38.328667 29185 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 15:04:38.328720 29185 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 15:04:38.328761 29185 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 15:04:38.328792 29185 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 15:04:38.334862 29185 rpc_server.cc:307] RPC server started. Bound to: 127.13.75.130:44675
I20260923 15:04:38.334937 29313 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.75.130:44675 every 8 connection(s)
I20260923 15:04:38.335278 29185 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/minicluster-data/ts-1/data/info.pb
Sep 23 15:04:38 dist-test-slave-cnb0 krb5kdc[28944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175878, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.130@KRBTEST.COM for kudu/127.13.75.190@KRBTEST.COM
I20260923 15:04:38.344146 13614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKC4soy/build/release/bin/kudu as pid 29185
I20260923 15:04:38.344264 13614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/minicluster-data/ts-1/wal/instance
I20260923 15:04:38.344493 13614 external_mini_cluster.cc:1428] Setting key 9200c28be53eada0d0cee82bba57954e
I20260923 15:04:38.345818 29034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:38.335969 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.130:42937 (local address 127.13.75.190:40941)
0923 15:04:38.336013 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0923 15:04:38.336015 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:04:38.336195 (+ 180us) server_negotiation.cc:407] Connection header received
0923 15:04:38.336738 (+ 543us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:04:38.336739 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:04:38.336769 (+ 30us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:04:38.336840 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:04:38.337862 (+ 1022us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:04:38.338462 (+ 600us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:04:38.339599 (+ 1137us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:04:38.339832 (+ 233us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:04:38.339897 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:04:38.342271 (+ 2374us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:04:38.342273 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:04:38.342274 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:04:38.342284 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:04:38.343795 (+ 1511us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:04:38.344296 (+ 501us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:04:38.344297 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:04:38.344298 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:04:38.344368 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:04:38.344745 (+ 377us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:04:38.344746 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:04:38.344747 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:04:38.344988 (+ 241us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:04:38.345054 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0923 15:04:38.345553 (+ 499us) server_negotiation.cc:299] Negotiation successful
0923 15:04:38.345675 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260923 15:04:38.346141 29316 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:38.335933 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:40941 (local address 127.13.75.130:42937)
0923 15:04:38.336082 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0923 15:04:38.336124 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0923 15:04:38.336607 (+ 483us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:04:38.336939 (+ 332us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:04:38.336943 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:04:38.337540 (+ 597us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:04:38.337752 (+ 212us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:04:38.337757 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:04:38.338560 (+ 803us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:04:38.338561 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:04:38.339510 (+ 949us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:04:38.339513 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:04:38.339917 (+ 404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:04:38.339918 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:04:38.340027 (+ 109us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:04:38.340615 (+ 588us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:04:38.340623 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:04:38.342180 (+ 1557us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:04:38.343887 (+ 1707us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:04:38.343895 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:04:38.343897 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:04:38.344163 (+ 266us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:04:38.344512 (+ 349us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:04:38.344513 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:04:38.344513 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:04:38.344664 (+ 151us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:04:38.345052 (+ 388us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:04:38.345060 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:04:38.345255 (+ 195us) client_negotiation.cc:769] Sending connection context
0923 15:04:38.345454 (+ 199us) client_negotiation.cc:240] Negotiation successful
0923 15:04:38.345658 (+ 204us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":80,"threads_started":1}
I20260923 15:04:38.346905 29314 heartbeater.cc:347] Connected to a master server at 127.13.75.190:40941
I20260923 15:04:38.347028 29314 heartbeater.cc:464] Registering TS with master...
I20260923 15:04:38.347218 29314 heartbeater.cc:511] Master 127.13.75.190:40941 requested a full tablet report, sending...
I20260923 15:04:38.347681 28989 ts_manager.cc:194] Registered new tserver with Master: fd45cc8453994d0fabe2a44f19a74598 (127.13.75.130:44675)
I20260923 15:04:38.348176 28989 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.75.130@KRBTEST.COM'} at 127.13.75.130:42937
WARNING: no policy specified for kudu/127.13.75.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.75.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.75.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.131.keytab.
Entry for principal kudu/127.13.75.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.131.keytab.
I20260923 15:04:38.371775 13614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKC4soy/build/release/bin/kudu
/tmp/dist-test-taskKC4soy/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.75.131:0
--local_ip_for_outbound_sockets=127.13.75.131
--webserver_interface=127.13.75.131
--webserver_port=0
--tserver_master_addrs=127.13.75.190:40941
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.131.keytab
--principal=kudu/127.13.75.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.75.148:37659
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:04:38.454246 29319 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:04:38.454454 29319 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:04:38.454479 29319 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:04:38.455629 29319 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:04:38.455664 29319 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:04:38.455690 29319 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.75.131
I20260923 15:04:38.457243 29319 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.75.148:37659
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.131.keytab
--principal=kudu/127.13.75.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.75.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.75.131
--webserver_port=0
--tserver_master_addrs=127.13.75.190:40941
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29319
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.75.131
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:04:38.457443 29319 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:04:38.457657 29319 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:04:38.462968 29324 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:04:38.462998 29327 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:04:38.463163 29319 server_base.cc:1034] running on GCE node
W20260923 15:04:38.462968 29325 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:04:38.463428 29319 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:04:38.463634 29319 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:04:38.464812 29319 hybrid_clock.cc:648] HybridClock initialized: now 1790175878464781 us; error 44 us; skew 500 ppm
Sep 23 15:04:38 dist-test-slave-cnb0 krb5kdc[28944](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175878, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:04:38.467424 29319 init.cc:377] Logged in from keytab as kudu/127.13.75.131@KRBTEST.COM (short username kudu)
I20260923 15:04:38.467873 29319 webserver.cc:460] Webserver started at http://127.13.75.131:42631/ using document root <none> and password file <none>
I20260923 15:04:38.468111 29319 fs_manager.cc:359] Metadata directory not provided
I20260923 15:04:38.468155 29319 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:04:38.468293 29319 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:04:38.469364 29319 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/minicluster-data/ts-2/data/instance:
uuid: "4c6169b10f2e4152be4f90c81836aadb"
format_stamp: "Formatted at 2026-09-23 15:04:38 on dist-test-slave-cnb0"
server_key: "0e829147c691b0c79bba8c0a6cab77a2"
server_key_iv: "930ca62486c0322d96007a120943a966"
server_key_version: "encryptionkey@0"
I20260923 15:04:38.469710 29319 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/minicluster-data/ts-2/wal/instance:
uuid: "4c6169b10f2e4152be4f90c81836aadb"
format_stamp: "Formatted at 2026-09-23 15:04:38 on dist-test-slave-cnb0"
server_key: "0e829147c691b0c79bba8c0a6cab77a2"
server_key_iv: "930ca62486c0322d96007a120943a966"
server_key_version: "encryptionkey@0"
I20260923 15:04:38.471124 29319 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 15:04:38.471937 29334 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:04:38.472149 29319 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:04:38.472230 29319 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/minicluster-data/ts-2/wal
uuid: "4c6169b10f2e4152be4f90c81836aadb"
format_stamp: "Formatted at 2026-09-23 15:04:38 on dist-test-slave-cnb0"
server_key: "0e829147c691b0c79bba8c0a6cab77a2"
server_key_iv: "930ca62486c0322d96007a120943a966"
server_key_version: "encryptionkey@0"
I20260923 15:04:38.472322 29319 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:04:38.488956 29319 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:04:38.491082 29319 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:04:38.491222 29319 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:04:38.491464 29319 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 15:04:38.491777 29319 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 15:04:38.491811 29319 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 15:04:38.491835 29319 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 15:04:38.491876 29319 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 15:04:38.497915 29319 rpc_server.cc:307] RPC server started. Bound to: 127.13.75.131:38233
I20260923 15:04:38.497998 29447 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.75.131:38233 every 8 connection(s)
I20260923 15:04:38.498297 29319 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/minicluster-data/ts-2/data/info.pb
Sep 23 15:04:38 dist-test-slave-cnb0 krb5kdc[28944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175878, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.131@KRBTEST.COM for kudu/127.13.75.190@KRBTEST.COM
I20260923 15:04:38.507381 13614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKC4soy/build/release/bin/kudu as pid 29319
I20260923 15:04:38.507483 13614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/minicluster-data/ts-2/wal/instance
I20260923 15:04:38.507669 13614 external_mini_cluster.cc:1428] Setting key 24a8bb6decbb9aedb190a62046815d88
I20260923 15:04:38.508227 29034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:38.499078 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.131:51635 (local address 127.13.75.190:40941)
0923 15:04:38.499123 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0923 15:04:38.499125 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:04:38.499376 (+ 251us) server_negotiation.cc:407] Connection header received
0923 15:04:38.499927 (+ 551us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:04:38.499929 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:04:38.499942 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:04:38.499993 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:04:38.500974 (+ 981us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:04:38.501512 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:04:38.502645 (+ 1133us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:04:38.502857 (+ 212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:04:38.503035 (+ 178us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:04:38.505293 (+ 2258us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:04:38.505295 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:04:38.505295 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:04:38.505302 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:04:38.506596 (+ 1294us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:04:38.507089 (+ 493us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:04:38.507090 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:04:38.507091 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:04:38.507155 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:04:38.507481 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:04:38.507482 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:04:38.507483 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:04:38.507632 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:04:38.507739 (+ 107us) server_negotiation.cc:1035] Waiting for connection context
0923 15:04:38.508047 (+ 308us) server_negotiation.cc:299] Negotiation successful
0923 15:04:38.508135 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260923 15:04:38.508471 29450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:38.499080 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:40941 (local address 127.13.75.131:51635)
0923 15:04:38.499252 (+ 172us) negotiation.cc:107] Waiting for socket to connect
0923 15:04:38.499289 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0923 15:04:38.499812 (+ 523us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:04:38.500073 (+ 261us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:04:38.500077 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:04:38.500667 (+ 590us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:04:38.500874 (+ 207us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:04:38.500879 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:04:38.501615 (+ 736us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:04:38.501617 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:04:38.502543 (+ 926us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:04:38.502546 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:04:38.503054 (+ 508us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:04:38.503056 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:04:38.503232 (+ 176us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:04:38.503758 (+ 526us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:04:38.503767 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:04:38.505197 (+ 1430us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:04:38.506680 (+ 1483us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:04:38.506688 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:04:38.506690 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:04:38.506997 (+ 307us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:04:38.507283 (+ 286us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:04:38.507283 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:04:38.507284 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:04:38.507391 (+ 107us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:04:38.507702 (+ 311us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:04:38.507709 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:04:38.507886 (+ 177us) client_negotiation.cc:769] Sending connection context
0923 15:04:38.508020 (+ 134us) client_negotiation.cc:240] Negotiation successful
0923 15:04:38.508186 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":87,"threads_started":1}
I20260923 15:04:38.508896 29448 heartbeater.cc:347] Connected to a master server at 127.13.75.190:40941
I20260923 15:04:38.509006 29448 heartbeater.cc:464] Registering TS with master...
I20260923 15:04:38.509203 29448 heartbeater.cc:511] Master 127.13.75.190:40941 requested a full tablet report, sending...
I20260923 15:04:38.509574 28989 ts_manager.cc:194] Registered new tserver with Master: 4c6169b10f2e4152be4f90c81836aadb (127.13.75.131:38233)
I20260923 15:04:38.509930 28989 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.75.131@KRBTEST.COM'} at 127.13.75.131:51635
I20260923 15:04:38.519601 13614 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260923 15:04:38.527251 29034 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:38.521002 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48776 (local address 127.13.75.190:40941)
0923 15:04:38.521044 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0923 15:04:38.521045 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 15:04:38.521191 (+ 146us) server_negotiation.cc:407] Connection header received
0923 15:04:38.521319 (+ 128us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:04:38.521320 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:04:38.521330 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:04:38.521380 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:04:38.521954 (+ 574us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:04:38.522628 (+ 674us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:04:38.523462 (+ 834us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:04:38.523655 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:04:38.523719 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:04:38.524862 (+ 1143us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:04:38.524863 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:04:38.524863 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:04:38.524870 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:04:38.526213 (+ 1343us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:04:38.526530 (+ 317us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:04:38.526531 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:04:38.526531 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:04:38.526580 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:04:38.526735 (+ 155us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:04:38.526736 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:04:38.526736 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:04:38.526945 (+ 209us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:04:38.527022 (+ 77us) server_negotiation.cc:1035] Waiting for connection context
0923 15:04:38.527099 (+ 77us) server_negotiation.cc:299] Negotiation successful
0923 15:04:38.527158 (+ 59us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260923 15:04:38.528766 28989 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48776:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 15:04:38.530393 28989 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 15:04:38.536901 29461 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:38.532678 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.130:44675 (local address 127.0.0.1:54430)
0923 15:04:38.532992 (+ 314us) negotiation.cc:107] Waiting for socket to connect
0923 15:04:38.533004 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 15:04:38.533052 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:04:38.533427 (+ 375us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:04:38.533431 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:04:38.533434 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:04:38.533521 (+ 87us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:04:38.533523 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:04:38.534506 (+ 983us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:04:38.534507 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:04:38.535884 (+ 1377us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:04:38.535886 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:04:38.536558 (+ 672us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:04:38.536559 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:04:38.536643 (+ 84us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:04:38.536656 (+ 13us) client_negotiation.cc:769] Sending connection context
0923 15:04:38.536743 (+ 87us) client_negotiation.cc:240] Negotiation successful
0923 15:04:38.536810 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":296,"mutex_wait_us":298,"thread_start_us":97,"threads_started":1}
I20260923 15:04:38.537066 29462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:38.532859 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40736 (local address 127.13.75.129:39471)
0923 15:04:38.533241 (+ 382us) server_negotiation.cc:206] Beginning negotiation
0923 15:04:38.533244 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:04:38.533254 (+ 10us) server_negotiation.cc:407] Connection header received
0923 15:04:38.533299 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:04:38.533301 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:04:38.533318 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:04:38.533596 (+ 278us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:04:38.533664 (+ 68us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:04:38.534333 (+ 669us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:04:38.535913 (+ 1580us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:04:38.536633 (+ 720us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:04:38.536717 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:04:38.536767 (+ 50us) server_negotiation.cc:1035] Waiting for connection context
0923 15:04:38.536869 (+ 102us) server_negotiation.cc:299] Negotiation successful
0923 15:04:38.536946 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":349,"thread_start_us":82,"threads_started":1}
I20260923 15:04:38.537212 29459 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:38.532660 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.129:39471 (local address 127.0.0.1:40736)
0923 15:04:38.532762 (+ 102us) negotiation.cc:107] Waiting for socket to connect
0923 15:04:38.532801 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0923 15:04:38.532864 (+ 63us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:04:38.533429 (+ 565us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:04:38.533432 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:04:38.533437 (+ 5us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:04:38.533513 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:04:38.533515 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:04:38.534427 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:04:38.534428 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:04:38.535809 (+ 1381us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:04:38.535812 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:04:38.536724 (+ 912us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:04:38.536725 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:04:38.536794 (+ 69us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:04:38.536797 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 15:04:38.536854 (+ 57us) client_negotiation.cc:240] Negotiation successful
0923 15:04:38.536888 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":77,"mutex_wait_us":50,"thread_start_us":57,"threads_started":1}
I20260923 15:04:38.537271 29464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:38.533037 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:54430 (local address 127.13.75.130:44675)
0923 15:04:38.533191 (+ 154us) server_negotiation.cc:206] Beginning negotiation
0923 15:04:38.533197 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 15:04:38.533208 (+ 11us) server_negotiation.cc:407] Connection header received
0923 15:04:38.533250 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:04:38.533252 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:04:38.533270 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:04:38.533406 (+ 136us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:04:38.533624 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:04:38.534376 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:04:38.535955 (+ 1579us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:04:38.536457 (+ 502us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:04:38.536553 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:04:38.536618 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0923 15:04:38.537118 (+ 500us) server_negotiation.cc:299] Negotiation successful
0923 15:04:38.537176 (+ 58us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":121,"thread_start_us":86,"threads_started":1}
I20260923 15:04:38.537976 29248 tablet_service.cc:1467] Processing CreateTablet for tablet aceb8ae41b9342c0bcb31a8feb501b71 (DEFAULT_TABLE table=test-table [id=b04244703ae64caca1848868211a7acc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 15:04:38.538331 29463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:38.532956 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39824 (local address 127.13.75.131:38233)
0923 15:04:38.533123 (+ 167us) server_negotiation.cc:206] Beginning negotiation
0923 15:04:38.533128 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 15:04:38.534523 (+ 1395us) server_negotiation.cc:407] Connection header received
0923 15:04:38.534594 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:04:38.534597 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:04:38.534612 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:04:38.534698 (+ 86us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:04:38.534794 (+ 96us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:04:38.535665 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:04:38.537064 (+ 1399us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:04:38.537902 (+ 838us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:04:38.537981 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:04:38.538033 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0923 15:04:38.538208 (+ 175us) server_negotiation.cc:299] Negotiation successful
0923 15:04:38.538252 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":101,"threads_started":1}
I20260923 15:04:38.538342 29460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:38.532877 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.131:38233 (local address 127.0.0.1:39824)
0923 15:04:38.534444 (+ 1567us) negotiation.cc:107] Waiting for socket to connect
0923 15:04:38.534452 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0923 15:04:38.534521 (+ 69us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:04:38.534695 (+ 174us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:04:38.534696 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:04:38.534698 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:04:38.534737 (+ 39us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:04:38.534739 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:04:38.535765 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:04:38.535766 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:04:38.536875 (+ 1109us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:04:38.536878 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:04:38.538005 (+ 1127us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:04:38.538006 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:04:38.538054 (+ 48us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:04:38.538059 (+ 5us) client_negotiation.cc:769] Sending connection context
0923 15:04:38.538182 (+ 123us) client_negotiation.cc:240] Negotiation successful
0923 15:04:38.538224 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":1549,"thread_start_us":65,"threads_started":1}
I20260923 15:04:38.538352 29248 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aceb8ae41b9342c0bcb31a8feb501b71. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:04:38.538466 29114 tablet_service.cc:1467] Processing CreateTablet for tablet aceb8ae41b9342c0bcb31a8feb501b71 (DEFAULT_TABLE table=test-table [id=b04244703ae64caca1848868211a7acc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 15:04:38.538746 29114 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aceb8ae41b9342c0bcb31a8feb501b71. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:04:38.538971 29382 tablet_service.cc:1467] Processing CreateTablet for tablet aceb8ae41b9342c0bcb31a8feb501b71 (DEFAULT_TABLE table=test-table [id=b04244703ae64caca1848868211a7acc]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 15:04:38.539203 29382 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet aceb8ae41b9342c0bcb31a8feb501b71. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:04:38.541774 29465 tablet_bootstrap.cc:492] T aceb8ae41b9342c0bcb31a8feb501b71 P 4c6169b10f2e4152be4f90c81836aadb: Bootstrap starting.
I20260923 15:04:38.542207 29466 tablet_bootstrap.cc:492] T aceb8ae41b9342c0bcb31a8feb501b71 P 125b34b86b954465bc1945b9733a2b0a: Bootstrap starting.
I20260923 15:04:38.542268 29467 tablet_bootstrap.cc:492] T aceb8ae41b9342c0bcb31a8feb501b71 P fd45cc8453994d0fabe2a44f19a74598: Bootstrap starting.
I20260923 15:04:38.542521 29465 tablet_bootstrap.cc:654] T aceb8ae41b9342c0bcb31a8feb501b71 P 4c6169b10f2e4152be4f90c81836aadb: Neither blocks nor log segments found. Creating new log.
I20260923 15:04:38.542841 29466 tablet_bootstrap.cc:654] T aceb8ae41b9342c0bcb31a8feb501b71 P 125b34b86b954465bc1945b9733a2b0a: Neither blocks nor log segments found. Creating new log.
I20260923 15:04:38.542982 29467 tablet_bootstrap.cc:654] T aceb8ae41b9342c0bcb31a8feb501b71 P fd45cc8453994d0fabe2a44f19a74598: Neither blocks nor log segments found. Creating new log.
I20260923 15:04:38.543157 29465 log.cc:824] T aceb8ae41b9342c0bcb31a8feb501b71 P 4c6169b10f2e4152be4f90c81836aadb: Log is configured to *not* fsync() on all Append() calls
I20260923 15:04:38.543480 29466 log.cc:824] T aceb8ae41b9342c0bcb31a8feb501b71 P 125b34b86b954465bc1945b9733a2b0a: Log is configured to *not* fsync() on all Append() calls
I20260923 15:04:38.543581 29467 log.cc:824] T aceb8ae41b9342c0bcb31a8feb501b71 P fd45cc8453994d0fabe2a44f19a74598: Log is configured to *not* fsync() on all Append() calls
I20260923 15:04:38.544068 29465 tablet_bootstrap.cc:492] T aceb8ae41b9342c0bcb31a8feb501b71 P 4c6169b10f2e4152be4f90c81836aadb: No bootstrap required, opened a new log
I20260923 15:04:38.544147 29465 ts_tablet_manager.cc:1397] T aceb8ae41b9342c0bcb31a8feb501b71 P 4c6169b10f2e4152be4f90c81836aadb: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260923 15:04:38.544203 29466 tablet_bootstrap.cc:492] T aceb8ae41b9342c0bcb31a8feb501b71 P 125b34b86b954465bc1945b9733a2b0a: No bootstrap required, opened a new log
I20260923 15:04:38.544281 29466 ts_tablet_manager.cc:1397] T aceb8ae41b9342c0bcb31a8feb501b71 P 125b34b86b954465bc1945b9733a2b0a: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260923 15:04:38.545463 29465 raft_consensus.cc:348] T aceb8ae41b9342c0bcb31a8feb501b71 P 4c6169b10f2e4152be4f90c81836aadb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "125b34b86b954465bc1945b9733a2b0a" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 39471 } } peers { permanent_uuid: "fd45cc8453994d0fabe2a44f19a74598" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 44675 } } peers { permanent_uuid: "4c6169b10f2e4152be4f90c81836aadb" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 38233 } }
I20260923 15:04:38.545619 29465 raft_consensus.cc:374] T aceb8ae41b9342c0bcb31a8feb501b71 P 4c6169b10f2e4152be4f90c81836aadb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:04:38.545668 29465 raft_consensus.cc:729] T aceb8ae41b9342c0bcb31a8feb501b71 P 4c6169b10f2e4152be4f90c81836aadb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c6169b10f2e4152be4f90c81836aadb, State: Initialized, Role: FOLLOWER
I20260923 15:04:38.545584 29466 raft_consensus.cc:348] T aceb8ae41b9342c0bcb31a8feb501b71 P 125b34b86b954465bc1945b9733a2b0a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "125b34b86b954465bc1945b9733a2b0a" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 39471 } } peers { permanent_uuid: "fd45cc8453994d0fabe2a44f19a74598" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 44675 } } peers { permanent_uuid: "4c6169b10f2e4152be4f90c81836aadb" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 38233 } }
I20260923 15:04:38.545754 29466 raft_consensus.cc:374] T aceb8ae41b9342c0bcb31a8feb501b71 P 125b34b86b954465bc1945b9733a2b0a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:04:38.545804 29466 raft_consensus.cc:729] T aceb8ae41b9342c0bcb31a8feb501b71 P 125b34b86b954465bc1945b9733a2b0a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 125b34b86b954465bc1945b9733a2b0a, State: Initialized, Role: FOLLOWER
I20260923 15:04:38.545791 29465 consensus_queue.cc:260] T aceb8ae41b9342c0bcb31a8feb501b71 P 4c6169b10f2e4152be4f90c81836aadb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "125b34b86b954465bc1945b9733a2b0a" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 39471 } } peers { permanent_uuid: "fd45cc8453994d0fabe2a44f19a74598" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 44675 } } peers { permanent_uuid: "4c6169b10f2e4152be4f90c81836aadb" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 38233 } }
I20260923 15:04:38.545931 29466 consensus_queue.cc:260] T aceb8ae41b9342c0bcb31a8feb501b71 P 125b34b86b954465bc1945b9733a2b0a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "125b34b86b954465bc1945b9733a2b0a" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 39471 } } peers { permanent_uuid: "fd45cc8453994d0fabe2a44f19a74598" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 44675 } } peers { permanent_uuid: "4c6169b10f2e4152be4f90c81836aadb" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 38233 } }
I20260923 15:04:38.546074 29465 ts_tablet_manager.cc:1428] T aceb8ae41b9342c0bcb31a8feb501b71 P 4c6169b10f2e4152be4f90c81836aadb: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260923 15:04:38.546080 29448 heartbeater.cc:503] Master 127.13.75.190:40941 was elected leader, sending a full tablet report...
I20260923 15:04:38.546204 29466 ts_tablet_manager.cc:1428] T aceb8ae41b9342c0bcb31a8feb501b71 P 125b34b86b954465bc1945b9733a2b0a: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260923 15:04:38.546334 29180 heartbeater.cc:503] Master 127.13.75.190:40941 was elected leader, sending a full tablet report...
I20260923 15:04:38.546753 29467 tablet_bootstrap.cc:492] T aceb8ae41b9342c0bcb31a8feb501b71 P fd45cc8453994d0fabe2a44f19a74598: No bootstrap required, opened a new log
I20260923 15:04:38.546828 29467 ts_tablet_manager.cc:1397] T aceb8ae41b9342c0bcb31a8feb501b71 P fd45cc8453994d0fabe2a44f19a74598: Time spent bootstrapping tablet: real 0.005s user 0.000s sys 0.002s
I20260923 15:04:38.548556 29467 raft_consensus.cc:348] T aceb8ae41b9342c0bcb31a8feb501b71 P fd45cc8453994d0fabe2a44f19a74598 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "125b34b86b954465bc1945b9733a2b0a" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 39471 } } peers { permanent_uuid: "fd45cc8453994d0fabe2a44f19a74598" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 44675 } } peers { permanent_uuid: "4c6169b10f2e4152be4f90c81836aadb" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 38233 } }
I20260923 15:04:38.548681 29467 raft_consensus.cc:374] T aceb8ae41b9342c0bcb31a8feb501b71 P fd45cc8453994d0fabe2a44f19a74598 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:04:38.548702 29467 raft_consensus.cc:729] T aceb8ae41b9342c0bcb31a8feb501b71 P fd45cc8453994d0fabe2a44f19a74598 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd45cc8453994d0fabe2a44f19a74598, State: Initialized, Role: FOLLOWER
I20260923 15:04:38.548832 29467 consensus_queue.cc:260] T aceb8ae41b9342c0bcb31a8feb501b71 P fd45cc8453994d0fabe2a44f19a74598 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "125b34b86b954465bc1945b9733a2b0a" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 39471 } } peers { permanent_uuid: "fd45cc8453994d0fabe2a44f19a74598" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 44675 } } peers { permanent_uuid: "4c6169b10f2e4152be4f90c81836aadb" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 38233 } }
I20260923 15:04:38.549118 29467 ts_tablet_manager.cc:1428] T aceb8ae41b9342c0bcb31a8feb501b71 P fd45cc8453994d0fabe2a44f19a74598: Time spent starting tablet: real 0.002s user 0.002s sys 0.001s
I20260923 15:04:38.549142 29314 heartbeater.cc:503] Master 127.13.75.190:40941 was elected leader, sending a full tablet report...
I20260923 15:04:38.570633 29471 raft_consensus.cc:482] T aceb8ae41b9342c0bcb31a8feb501b71 P 4c6169b10f2e4152be4f90c81836aadb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 15:04:38.570788 29471 raft_consensus.cc:504] T aceb8ae41b9342c0bcb31a8feb501b71 P 4c6169b10f2e4152be4f90c81836aadb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "125b34b86b954465bc1945b9733a2b0a" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 39471 } } peers { permanent_uuid: "fd45cc8453994d0fabe2a44f19a74598" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 44675 } } peers { permanent_uuid: "4c6169b10f2e4152be4f90c81836aadb" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 38233 } }
I20260923 15:04:38.571219 29471 leader_election.cc:288] T aceb8ae41b9342c0bcb31a8feb501b71 P 4c6169b10f2e4152be4f90c81836aadb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 125b34b86b954465bc1945b9733a2b0a (127.13.75.129:39471), fd45cc8453994d0fabe2a44f19a74598 (127.13.75.130:44675)
I20260923 15:04:38.574002 29450 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:38.571417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.129:39471 (local address 127.13.75.131:45949)
0923 15:04:38.571482 (+ 65us) negotiation.cc:107] Waiting for socket to connect
0923 15:04:38.571496 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0923 15:04:38.571554 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:04:38.571713 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:04:38.571714 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:04:38.571715 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:04:38.571793 (+ 78us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:04:38.571808 (+ 15us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:04:38.572446 (+ 638us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:04:38.572447 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:04:38.573190 (+ 743us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:04:38.573192 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:04:38.573750 (+ 558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:04:38.573751 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:04:38.573830 (+ 79us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:04:38.573833 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 15:04:38.573894 (+ 61us) client_negotiation.cc:240] Negotiation successful
0923 15:04:38.573934 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":42}
I20260923 15:04:38.574002 29462 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:38.571539 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.131:45949 (local address 127.13.75.129:39471)
0923 15:04:38.571590 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0923 15:04:38.571592 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:04:38.571601 (+ 9us) server_negotiation.cc:407] Connection header received
0923 15:04:38.571629 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:04:38.571629 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:04:38.571640 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:04:38.571703 (+ 63us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:04:38.571883 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:04:38.572356 (+ 473us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:04:38.573287 (+ 931us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:04:38.573664 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:04:38.573741 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:04:38.573750 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0923 15:04:38.573900 (+ 150us) server_negotiation.cc:299] Negotiation successful
0923 15:04:38.573936 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260923 15:04:38.574141 29474 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:38.571417 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.130:44675 (local address 127.13.75.131:42985)
0923 15:04:38.571684 (+ 267us) negotiation.cc:107] Waiting for socket to connect
0923 15:04:38.571691 (+ 7us) client_negotiation.cc:174] Beginning negotiation
0923 15:04:38.571746 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:04:38.571874 (+ 128us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:04:38.571874 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:04:38.571876 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:04:38.571946 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:04:38.571948 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:04:38.572576 (+ 628us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:04:38.572577 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:04:38.573325 (+ 748us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:04:38.573327 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:04:38.573878 (+ 551us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:04:38.573879 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:04:38.573950 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:04:38.573954 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 15:04:38.574017 (+ 63us) client_negotiation.cc:240] Negotiation successful
0923 15:04:38.574055 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":250,"mutex_wait_us":92,"thread_start_us":126,"threads_started":1}
I20260923 15:04:38.574129 29464 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:38.571500 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.131:42985 (local address 127.13.75.130:44675)
0923 15:04:38.571559 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0923 15:04:38.571561 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:04:38.571743 (+ 182us) server_negotiation.cc:407] Connection header received
0923 15:04:38.571799 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:04:38.571800 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:04:38.571810 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:04:38.571872 (+ 62us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:04:38.572017 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:04:38.572494 (+ 477us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:04:38.573417 (+ 923us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:04:38.573798 (+ 381us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:04:38.573878 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:04:38.573886 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0923 15:04:38.574035 (+ 149us) server_negotiation.cc:299] Negotiation successful
0923 15:04:38.574070 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260923 15:04:38.574354 29134 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aceb8ae41b9342c0bcb31a8feb501b71" candidate_uuid: "4c6169b10f2e4152be4f90c81836aadb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "125b34b86b954465bc1945b9733a2b0a" is_pre_election: true
I20260923 15:04:38.574395 29268 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aceb8ae41b9342c0bcb31a8feb501b71" candidate_uuid: "4c6169b10f2e4152be4f90c81836aadb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd45cc8453994d0fabe2a44f19a74598" is_pre_election: true
I20260923 15:04:38.574520 29268 raft_consensus.cc:2454] T aceb8ae41b9342c0bcb31a8feb501b71 P fd45cc8453994d0fabe2a44f19a74598 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4c6169b10f2e4152be4f90c81836aadb in term 0.
I20260923 15:04:38.574522 29134 raft_consensus.cc:2454] T aceb8ae41b9342c0bcb31a8feb501b71 P 125b34b86b954465bc1945b9733a2b0a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4c6169b10f2e4152be4f90c81836aadb in term 0.
I20260923 15:04:38.574728 29338 leader_election.cc:302] T aceb8ae41b9342c0bcb31a8feb501b71 P 4c6169b10f2e4152be4f90c81836aadb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c6169b10f2e4152be4f90c81836aadb, fd45cc8453994d0fabe2a44f19a74598; no voters:
I20260923 15:04:38.574859 29471 raft_consensus.cc:2781] T aceb8ae41b9342c0bcb31a8feb501b71 P 4c6169b10f2e4152be4f90c81836aadb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 15:04:38.574935 29471 raft_consensus.cc:482] T aceb8ae41b9342c0bcb31a8feb501b71 P 4c6169b10f2e4152be4f90c81836aadb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 15:04:38.574978 29471 raft_consensus.cc:3037] T aceb8ae41b9342c0bcb31a8feb501b71 P 4c6169b10f2e4152be4f90c81836aadb [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:04:38.575966 29471 raft_consensus.cc:504] T aceb8ae41b9342c0bcb31a8feb501b71 P 4c6169b10f2e4152be4f90c81836aadb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "125b34b86b954465bc1945b9733a2b0a" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 39471 } } peers { permanent_uuid: "fd45cc8453994d0fabe2a44f19a74598" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 44675 } } peers { permanent_uuid: "4c6169b10f2e4152be4f90c81836aadb" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 38233 } }
I20260923 15:04:38.576112 29471 leader_election.cc:288] T aceb8ae41b9342c0bcb31a8feb501b71 P 4c6169b10f2e4152be4f90c81836aadb [CANDIDATE]: Term 1 election: Requested vote from peers 125b34b86b954465bc1945b9733a2b0a (127.13.75.129:39471), fd45cc8453994d0fabe2a44f19a74598 (127.13.75.130:44675)
I20260923 15:04:38.576269 29134 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aceb8ae41b9342c0bcb31a8feb501b71" candidate_uuid: "4c6169b10f2e4152be4f90c81836aadb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "125b34b86b954465bc1945b9733a2b0a"
I20260923 15:04:38.576287 29268 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "aceb8ae41b9342c0bcb31a8feb501b71" candidate_uuid: "4c6169b10f2e4152be4f90c81836aadb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd45cc8453994d0fabe2a44f19a74598"
I20260923 15:04:38.576367 29134 raft_consensus.cc:3037] T aceb8ae41b9342c0bcb31a8feb501b71 P 125b34b86b954465bc1945b9733a2b0a [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:04:38.576362 29268 raft_consensus.cc:3037] T aceb8ae41b9342c0bcb31a8feb501b71 P fd45cc8453994d0fabe2a44f19a74598 [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:04:38.577046 29268 raft_consensus.cc:2454] T aceb8ae41b9342c0bcb31a8feb501b71 P fd45cc8453994d0fabe2a44f19a74598 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4c6169b10f2e4152be4f90c81836aadb in term 1.
I20260923 15:04:38.577174 29134 raft_consensus.cc:2454] T aceb8ae41b9342c0bcb31a8feb501b71 P 125b34b86b954465bc1945b9733a2b0a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4c6169b10f2e4152be4f90c81836aadb in term 1.
I20260923 15:04:38.577198 29338 leader_election.cc:302] T aceb8ae41b9342c0bcb31a8feb501b71 P 4c6169b10f2e4152be4f90c81836aadb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4c6169b10f2e4152be4f90c81836aadb, fd45cc8453994d0fabe2a44f19a74598; no voters:
I20260923 15:04:38.577301 29471 raft_consensus.cc:2781] T aceb8ae41b9342c0bcb31a8feb501b71 P 4c6169b10f2e4152be4f90c81836aadb [term 1 FOLLOWER]: Leader election won for term 1
I20260923 15:04:38.577469 29471 raft_consensus.cc:686] T aceb8ae41b9342c0bcb31a8feb501b71 P 4c6169b10f2e4152be4f90c81836aadb [term 1 LEADER]: Becoming Leader. State: Replica: 4c6169b10f2e4152be4f90c81836aadb, State: Running, Role: LEADER
I20260923 15:04:38.577565 29471 consensus_queue.cc:237] T aceb8ae41b9342c0bcb31a8feb501b71 P 4c6169b10f2e4152be4f90c81836aadb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "125b34b86b954465bc1945b9733a2b0a" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 39471 } } peers { permanent_uuid: "fd45cc8453994d0fabe2a44f19a74598" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 44675 } } peers { permanent_uuid: "4c6169b10f2e4152be4f90c81836aadb" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 38233 } }
I20260923 15:04:38.578372 28988 catalog_manager.cc:5495] T aceb8ae41b9342c0bcb31a8feb501b71 P 4c6169b10f2e4152be4f90c81836aadb reported cstate change: term changed from 0 to 1, leader changed from <none> to 4c6169b10f2e4152be4f90c81836aadb (127.13.75.131). New cstate: current_term: 1 leader_uuid: "4c6169b10f2e4152be4f90c81836aadb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "125b34b86b954465bc1945b9733a2b0a" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 39471 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "fd45cc8453994d0fabe2a44f19a74598" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 44675 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4c6169b10f2e4152be4f90c81836aadb" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 38233 } health_report { overall_health: HEALTHY } } }
W20260923 15:04:38.585798 29315 tablet.cc:2369] T aceb8ae41b9342c0bcb31a8feb501b71 P fd45cc8453994d0fabe2a44f19a74598: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 15:04:38.595530 29463 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:38.592735 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39834 (local address 127.13.75.131:38233)
0923 15:04:38.592802 (+ 67us) server_negotiation.cc:206] Beginning negotiation
0923 15:04:38.592804 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:04:38.592831 (+ 27us) server_negotiation.cc:407] Connection header received
0923 15:04:38.592866 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:04:38.592867 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:04:38.592883 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:04:38.592936 (+ 53us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 15:04:38.593222 (+ 286us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:04:38.593725 (+ 503us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:04:38.594550 (+ 825us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:04:38.594742 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:04:38.594819 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:04:38.595060 (+ 241us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 15:04:38.595169 (+ 109us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 15:04:38.595247 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0923 15:04:38.595351 (+ 104us) server_negotiation.cc:299] Negotiation successful
0923 15:04:38.595373 (+ 22us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260923 15:04:38.597620 29268 raft_consensus.cc:1261] T aceb8ae41b9342c0bcb31a8feb501b71 P fd45cc8453994d0fabe2a44f19a74598 [term 1 FOLLOWER]: Refusing update from remote peer 4c6169b10f2e4152be4f90c81836aadb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 15:04:38.597697 29134 raft_consensus.cc:1261] T aceb8ae41b9342c0bcb31a8feb501b71 P 125b34b86b954465bc1945b9733a2b0a [term 1 FOLLOWER]: Refusing update from remote peer 4c6169b10f2e4152be4f90c81836aadb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 15:04:38.598081 29475 consensus_queue.cc:1035] T aceb8ae41b9342c0bcb31a8feb501b71 P 4c6169b10f2e4152be4f90c81836aadb [LEADER]: Connected to new peer: Peer: permanent_uuid: "fd45cc8453994d0fabe2a44f19a74598" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 44675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 15:04:38.598237 29471 consensus_queue.cc:1035] T aceb8ae41b9342c0bcb31a8feb501b71 P 4c6169b10f2e4152be4f90c81836aadb [LEADER]: Connected to new peer: Peer: permanent_uuid: "125b34b86b954465bc1945b9733a2b0a" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 39471 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 15:04:38.600732 29478 mvcc.cc:205] Tried to move back new op lower bound from 7332560398732730368 to 7332560398654418944. Current Snapshot: MvccSnapshot[applied={T|T < 7332560398732730368}]
I20260923 15:04:38.601052 29479 mvcc.cc:205] Tried to move back new op lower bound from 7332560398732730368 to 7332560398654418944. Current Snapshot: MvccSnapshot[applied={T|T < 7332560398732730368}]
I20260923 15:04:38.601886 29480 mvcc.cc:205] Tried to move back new op lower bound from 7332560398732730368 to 7332560398654418944. Current Snapshot: MvccSnapshot[applied={T|T < 7332560398732730368 or (T in {7332560398732730368})}]
I20260923 15:04:38.604071 28988 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48776:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 15:04:38.604179 28988 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:48776:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 15:04:38.605018 28988 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 842c3ea43bfc4cda9118b7f7d8dd212c: Sending DeleteTablet for 3 replicas of tablet aceb8ae41b9342c0bcb31a8feb501b71
I20260923 15:04:38.605427 29114 tablet_service.cc:1514] Processing DeleteTablet for tablet aceb8ae41b9342c0bcb31a8feb501b71 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 15:04:38 UTC) from {username='kudu', principal='kudu/127.13.75.190@KRBTEST.COM'} at 127.0.0.1:40736
I20260923 15:04:38.605494 29248 tablet_service.cc:1514] Processing DeleteTablet for tablet aceb8ae41b9342c0bcb31a8feb501b71 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 15:04:38 UTC) from {username='kudu', principal='kudu/127.13.75.190@KRBTEST.COM'} at 127.0.0.1:54430
I20260923 15:04:38.605541 29382 tablet_service.cc:1514] Processing DeleteTablet for tablet aceb8ae41b9342c0bcb31a8feb501b71 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 15:04:38 UTC) from {username='kudu', principal='kudu/127.13.75.190@KRBTEST.COM'} at 127.0.0.1:39824
I20260923 15:04:38.605724 29488 tablet_replica.cc:330] T aceb8ae41b9342c0bcb31a8feb501b71 P fd45cc8453994d0fabe2a44f19a74598: stopping tablet replica
I20260923 15:04:38.605834 13614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKC4soy/build/release/bin/kudu with pid 29051
I20260923 15:04:38.605860 29488 raft_consensus.cc:2229] T aceb8ae41b9342c0bcb31a8feb501b71 P fd45cc8453994d0fabe2a44f19a74598 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 15:04:38.606436 29488 raft_consensus.cc:2258] T aceb8ae41b9342c0bcb31a8feb501b71 P fd45cc8453994d0fabe2a44f19a74598 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 15:04:38.606942 29488 ts_tablet_manager.cc:1905] T aceb8ae41b9342c0bcb31a8feb501b71 P fd45cc8453994d0fabe2a44f19a74598: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 15:04:38.608673 29489 tablet_replica.cc:330] T aceb8ae41b9342c0bcb31a8feb501b71 P 4c6169b10f2e4152be4f90c81836aadb: stopping tablet replica
I20260923 15:04:38.608822 29488 ts_tablet_manager.cc:1918] T aceb8ae41b9342c0bcb31a8feb501b71 P fd45cc8453994d0fabe2a44f19a74598: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 15:04:38.608911 29488 log.cc:1196] T aceb8ae41b9342c0bcb31a8feb501b71 P fd45cc8453994d0fabe2a44f19a74598: Deleting WAL directory at /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/minicluster-data/ts-1/wal/wals/aceb8ae41b9342c0bcb31a8feb501b71
I20260923 15:04:38.609169 29488 ts_tablet_manager.cc:1939] T aceb8ae41b9342c0bcb31a8feb501b71 P fd45cc8453994d0fabe2a44f19a74598: Deleting consensus metadata
I20260923 15:04:38.609764 29489 raft_consensus.cc:2229] T aceb8ae41b9342c0bcb31a8feb501b71 P 4c6169b10f2e4152be4f90c81836aadb [term 1 LEADER]: Raft consensus shutting down.
I20260923 15:04:38.609788 28976 catalog_manager.cc:4841] TS fd45cc8453994d0fabe2a44f19a74598 (127.13.75.130:44675): tablet aceb8ae41b9342c0bcb31a8feb501b71 (table test-table [id=b04244703ae64caca1848868211a7acc]) successfully deleted
I20260923 15:04:38.609922 29489 raft_consensus.cc:2258] T aceb8ae41b9342c0bcb31a8feb501b71 P 4c6169b10f2e4152be4f90c81836aadb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 15:04:38.610296 29489 ts_tablet_manager.cc:1905] T aceb8ae41b9342c0bcb31a8feb501b71 P 4c6169b10f2e4152be4f90c81836aadb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 15:04:38.611639 29489 ts_tablet_manager.cc:1918] T aceb8ae41b9342c0bcb31a8feb501b71 P 4c6169b10f2e4152be4f90c81836aadb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 15:04:38.611724 29489 log.cc:1196] T aceb8ae41b9342c0bcb31a8feb501b71 P 4c6169b10f2e4152be4f90c81836aadb: Deleting WAL directory at /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/minicluster-data/ts-2/wal/wals/aceb8ae41b9342c0bcb31a8feb501b71
I20260923 15:04:38.611995 29489 ts_tablet_manager.cc:1939] T aceb8ae41b9342c0bcb31a8feb501b71 P 4c6169b10f2e4152be4f90c81836aadb: Deleting consensus metadata
I20260923 15:04:38.612401 28976 catalog_manager.cc:4841] TS 4c6169b10f2e4152be4f90c81836aadb (127.13.75.131:38233): tablet aceb8ae41b9342c0bcb31a8feb501b71 (table test-table [id=b04244703ae64caca1848868211a7acc]) successfully deleted
W20260923 15:04:38.613034 28975 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.75.129:39471 (error 108)
I20260923 15:04:38.613209 13614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKC4soy/build/release/bin/kudu with pid 29185
I20260923 15:04:38.618180 29460 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:38.618020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.129:39471 (local address 127.0.0.1:40752)
0923 15:04:38.618086 (+ 66us) negotiation.cc:107] Waiting for socket to connect
0923 15:04:38.618141 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.129:39471: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":41}
I20260923 15:04:38.618239 13614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKC4soy/build/release/bin/kudu with pid 29319
W20260923 15:04:38.619586 28975 catalog_manager.cc:4584] TS 125b34b86b954465bc1945b9733a2b0a (127.13.75.129:39471): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet aceb8ae41b9342c0bcb31a8feb501b71: Network error: Client connection negotiation failed: client connection to 127.13.75.129:39471: connect: Connection refused (error 111)
I20260923 15:04:38.622943 13614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKC4soy/build/release/bin/kudu with pid 28957
I20260923 15:04:58.628337 13614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKC4soy/build/release/bin/kudu
/tmp/dist-test-taskKC4soy/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.75.190:40941
--webserver_interface=127.13.75.190
--webserver_port=35917
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.190.keytab
--principal=kudu/127.13.75.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.75.148:37659
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.75.190:40941
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--tsk_private_key_password_cmd=echo foo with env {KRB5CCNAME=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:04:58.713200 29491 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:04:58.713423 29491 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:04:58.713445 29491 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:04:58.714624 29491 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260923 15:04:58.714668 29491 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260923 15:04:58.714680 29491 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260923 15:04:58.714692 29491 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 15:04:58.714705 29491 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:04:58.714720 29491 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:04:58.714781 29491 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 15:04:58.714797 29491 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 15:04:58.716398 29491 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.75.148:37659
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_private_key_password_cmd=echo foo
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.13.75.190:40941
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.190.keytab
--principal=kudu/127.13.75.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.75.190:40941
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.75.190
--webserver_port=35917
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29491
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:04:58.716676 29491 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:04:58.716926 29491 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:04:58.726795 29497 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:04:58.726861 29498 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:04:58.726835 29491 server_base.cc:1034] running on GCE node
W20260923 15:04:58.726795 29500 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:04:58.727725 29491 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:04:58.728103 29491 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:04:58.729262 29491 hybrid_clock.cc:648] HybridClock initialized: now 1790175898729229 us; error 48 us; skew 500 ppm
Sep 23 15:04:58 dist-test-slave-cnb0 krb5kdc[28944](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175898, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:04:58.732373 29491 init.cc:377] Logged in from keytab as kudu/127.13.75.190@KRBTEST.COM (short username kudu)
I20260923 15:04:58.732887 29491 webserver.cc:460] Webserver started at http://127.13.75.190:35917/ using document root <none> and password file <none>
I20260923 15:04:58.733137 29491 fs_manager.cc:359] Metadata directory not provided
I20260923 15:04:58.733220 29491 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:04:58.734704 29491 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 15:04:58.735638 29507 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:04:58.735914 29491 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260923 15:04:58.736061 29491 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/minicluster-data/master-0/data,/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/minicluster-data/master-0/wal
uuid: "842c3ea43bfc4cda9118b7f7d8dd212c"
format_stamp: "Formatted at 2026-09-23 15:04:37 on dist-test-slave-cnb0"
server_key: "c5bd9a1448bd7566ed2d489c8e7943bb"
server_key_iv: "59b41eac661e9094ff8487caada62cc3"
server_key_version: "encryptionkey@0"
I20260923 15:04:58.736610 29491 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:04:58.761371 29491 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:04:58.763700 29491 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:04:58.763873 29491 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:04:58.768939 29491 rpc_server.cc:307] RPC server started. Bound to: 127.13.75.190:40941
I20260923 15:04:58.768983 29559 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.75.190:40941 every 8 connection(s)
I20260923 15:04:58.769431 29491 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/minicluster-data/master-0/data/info.pb
I20260923 15:04:58.771087 29560 sys_catalog.cc:263] Verifying existing consensus state
I20260923 15:04:58.771791 29560 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 842c3ea43bfc4cda9118b7f7d8dd212c: Bootstrap starting.
I20260923 15:04:58.773835 13614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKC4soy/build/release/bin/kudu as pid 29491
I20260923 15:04:58.774247 13614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKC4soy/build/release/bin/kudu
/tmp/dist-test-taskKC4soy/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.75.129:39471
--local_ip_for_outbound_sockets=127.13.75.129
--tserver_master_addrs=127.13.75.190:40941
--webserver_port=41567
--webserver_interface=127.13.75.129
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.129.keytab
--principal=kudu/127.13.75.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.75.148:37659
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260923 15:04:58.783706 29560 log.cc:824] T 00000000000000000000000000000000 P 842c3ea43bfc4cda9118b7f7d8dd212c: Log is configured to *not* fsync() on all Append() calls
I20260923 15:04:58.785732 29560 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 842c3ea43bfc4cda9118b7f7d8dd212c: Bootstrap replayed 1/1 log segments. Stats: ops{read=8 overwritten=0 applied=8 ignored=0} inserts{seen=5 ignored=0} mutations{seen=4 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260923 15:04:58.786070 29560 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 842c3ea43bfc4cda9118b7f7d8dd212c: Bootstrap complete.
I20260923 15:04:58.788516 29560 raft_consensus.cc:348] T 00000000000000000000000000000000 P 842c3ea43bfc4cda9118b7f7d8dd212c [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "842c3ea43bfc4cda9118b7f7d8dd212c" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 40941 } }
I20260923 15:04:58.788744 29560 raft_consensus.cc:729] T 00000000000000000000000000000000 P 842c3ea43bfc4cda9118b7f7d8dd212c [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 842c3ea43bfc4cda9118b7f7d8dd212c, State: Initialized, Role: FOLLOWER
I20260923 15:04:58.788913 29560 consensus_queue.cc:260] T 00000000000000000000000000000000 P 842c3ea43bfc4cda9118b7f7d8dd212c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "842c3ea43bfc4cda9118b7f7d8dd212c" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 40941 } }
I20260923 15:04:58.788983 29560 raft_consensus.cc:388] T 00000000000000000000000000000000 P 842c3ea43bfc4cda9118b7f7d8dd212c [term 1 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 15:04:58.789049 29560 raft_consensus.cc:482] T 00000000000000000000000000000000 P 842c3ea43bfc4cda9118b7f7d8dd212c [term 1 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 15:04:58.789103 29560 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 842c3ea43bfc4cda9118b7f7d8dd212c [term 1 FOLLOWER]: Advancing to term 2
I20260923 15:04:58.790151 29560 raft_consensus.cc:504] T 00000000000000000000000000000000 P 842c3ea43bfc4cda9118b7f7d8dd212c [term 2 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "842c3ea43bfc4cda9118b7f7d8dd212c" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 40941 } }
I20260923 15:04:58.790266 29560 leader_election.cc:302] T 00000000000000000000000000000000 P 842c3ea43bfc4cda9118b7f7d8dd212c [CANDIDATE]: Term 2 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 842c3ea43bfc4cda9118b7f7d8dd212c; no voters:
I20260923 15:04:58.790458 29560 leader_election.cc:288] T 00000000000000000000000000000000 P 842c3ea43bfc4cda9118b7f7d8dd212c [CANDIDATE]: Term 2 election: Requested vote from peers
I20260923 15:04:58.790494 29564 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 842c3ea43bfc4cda9118b7f7d8dd212c [term 2 FOLLOWER]: Leader election won for term 2
I20260923 15:04:58.790625 29564 raft_consensus.cc:686] T 00000000000000000000000000000000 P 842c3ea43bfc4cda9118b7f7d8dd212c [term 2 LEADER]: Becoming Leader. State: Replica: 842c3ea43bfc4cda9118b7f7d8dd212c, State: Running, Role: LEADER
I20260923 15:04:58.790711 29564 consensus_queue.cc:237] T 00000000000000000000000000000000 P 842c3ea43bfc4cda9118b7f7d8dd212c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 8, Committed index: 8, Last appended: 1.8, Last appended by leader: 8, Current term: 2, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "842c3ea43bfc4cda9118b7f7d8dd212c" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 40941 } }
I20260923 15:04:58.790738 29560 sys_catalog.cc:564] T 00000000000000000000000000000000 P 842c3ea43bfc4cda9118b7f7d8dd212c [sys.catalog]: configured and running, proceeding with master startup.
I20260923 15:04:58.791046 29566 sys_catalog.cc:455] T 00000000000000000000000000000000 P 842c3ea43bfc4cda9118b7f7d8dd212c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 2 leader_uuid: "842c3ea43bfc4cda9118b7f7d8dd212c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "842c3ea43bfc4cda9118b7f7d8dd212c" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 40941 } } }
I20260923 15:04:58.791049 29564 sys_catalog.cc:455] T 00000000000000000000000000000000 P 842c3ea43bfc4cda9118b7f7d8dd212c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 842c3ea43bfc4cda9118b7f7d8dd212c. Latest consensus state: current_term: 2 leader_uuid: "842c3ea43bfc4cda9118b7f7d8dd212c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "842c3ea43bfc4cda9118b7f7d8dd212c" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 40941 } } }
I20260923 15:04:58.791355 29564 sys_catalog.cc:458] T 00000000000000000000000000000000 P 842c3ea43bfc4cda9118b7f7d8dd212c [sys.catalog]: This master's current role is: LEADER
I20260923 15:04:58.791576 29566 sys_catalog.cc:458] T 00000000000000000000000000000000 P 842c3ea43bfc4cda9118b7f7d8dd212c [sys.catalog]: This master's current role is: LEADER
I20260923 15:04:58.791728 29571 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 15:04:58.792608 29571 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 15:04:58.792762 29571 catalog_manager.cc:1259] Loaded cluster ID: 5e9ad76930f74565ae7880afecf8f882
I20260923 15:04:58.792997 29571 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 15:04:58.794310 29571 catalog_manager.cc:1504] Loading token signing keys...
I20260923 15:04:58.794500 29571 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 842c3ea43bfc4cda9118b7f7d8dd212c: Loaded TSK: 0
I20260923 15:04:58.804225 29571 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 842c3ea43bfc4cda9118b7f7d8dd212c: Generated new TSK 1
I20260923 15:04:58.804332 29571 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260923 15:04:58.860491 29562 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:04:58.860822 29562 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:04:58.860865 29562 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:04:58.862102 29562 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:04:58.862164 29562 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:04:58.862211 29562 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.75.129
I20260923 15:04:58.863752 29562 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.75.148:37659
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.129.keytab
--principal=kudu/127.13.75.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.75.129:39471
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.75.129
--webserver_port=41567
--tserver_master_addrs=127.13.75.190:40941
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29562
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.75.129
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:04:58.864022 29562 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:04:58.864244 29562 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:04:58.869925 29588 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:04:58.869974 29587 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:04:58.870095 29590 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:04:58.870309 29562 server_base.cc:1034] running on GCE node
I20260923 15:04:58.870536 29562 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:04:58.870784 29562 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:04:58.871969 29562 hybrid_clock.cc:648] HybridClock initialized: now 1790175898871953 us; error 32 us; skew 500 ppm
Sep 23 15:04:58 dist-test-slave-cnb0 krb5kdc[28944](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175898, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:04:58.874785 29562 init.cc:377] Logged in from keytab as kudu/127.13.75.129@KRBTEST.COM (short username kudu)
I20260923 15:04:58.875329 29562 webserver.cc:460] Webserver started at http://127.13.75.129:41567/ using document root <none> and password file <none>
I20260923 15:04:58.875568 29562 fs_manager.cc:359] Metadata directory not provided
I20260923 15:04:58.875648 29562 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:04:58.877106 29562 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 15:04:58.877837 29597 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:04:58.878019 29562 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:04:58.878114 29562 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/minicluster-data/ts-0/wal
uuid: "125b34b86b954465bc1945b9733a2b0a"
format_stamp: "Formatted at 2026-09-23 15:04:38 on dist-test-slave-cnb0"
server_key: "30fb4b9008d66fe51a6d9aab609f2492"
server_key_iv: "41a52838ed269bdd2774e65fb947e496"
server_key_version: "encryptionkey@0"
I20260923 15:04:58.878479 29562 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:04:58.890046 29562 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:04:58.892822 29562 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:04:58.892989 29562 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:04:58.893266 29562 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 15:04:58.893858 29604 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260923 15:04:58.895612 29562 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260923 15:04:58.895668 29562 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260923 15:04:58.895702 29562 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260923 15:04:58.896473 29562 ts_tablet_manager.cc:610] Registered 1 tablets
I20260923 15:04:58.896520 29562 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260923 15:04:58.896572 29604 tablet_bootstrap.cc:492] T aceb8ae41b9342c0bcb31a8feb501b71 P 125b34b86b954465bc1945b9733a2b0a: Bootstrap starting.
I20260923 15:04:58.906076 29562 rpc_server.cc:307] RPC server started. Bound to: 127.13.75.129:39471
I20260923 15:04:58.906275 29711 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.75.129:39471 every 8 connection(s)
I20260923 15:04:58.906736 29562 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/minicluster-data/ts-0/data/info.pb
I20260923 15:04:58.910142 13614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKC4soy/build/release/bin/kudu as pid 29562
I20260923 15:04:58.910454 13614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKC4soy/build/release/bin/kudu
/tmp/dist-test-taskKC4soy/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.75.130:44675
--local_ip_for_outbound_sockets=127.13.75.130
--tserver_master_addrs=127.13.75.190:40941
--webserver_port=46595
--webserver_interface=127.13.75.130
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.130.keytab
--principal=kudu/127.13.75.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.75.148:37659
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260923 15:04:58.913378 29604 log.cc:824] T aceb8ae41b9342c0bcb31a8feb501b71 P 125b34b86b954465bc1945b9733a2b0a: Log is configured to *not* fsync() on all Append() calls
I20260923 15:04:58.916028 29604 tablet_bootstrap.cc:492] T aceb8ae41b9342c0bcb31a8feb501b71 P 125b34b86b954465bc1945b9733a2b0a: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260923 15:04:58.916342 29604 tablet_bootstrap.cc:492] T aceb8ae41b9342c0bcb31a8feb501b71 P 125b34b86b954465bc1945b9733a2b0a: Bootstrap complete.
I20260923 15:04:58.917606 29604 ts_tablet_manager.cc:1397] T aceb8ae41b9342c0bcb31a8feb501b71 P 125b34b86b954465bc1945b9733a2b0a: Time spent bootstrapping tablet: real 0.021s user 0.014s sys 0.004s
I20260923 15:04:58.918941 29604 raft_consensus.cc:348] T aceb8ae41b9342c0bcb31a8feb501b71 P 125b34b86b954465bc1945b9733a2b0a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "125b34b86b954465bc1945b9733a2b0a" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 39471 } } peers { permanent_uuid: "fd45cc8453994d0fabe2a44f19a74598" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 44675 } } peers { permanent_uuid: "4c6169b10f2e4152be4f90c81836aadb" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 38233 } }
I20260923 15:04:58.919252 29604 raft_consensus.cc:729] T aceb8ae41b9342c0bcb31a8feb501b71 P 125b34b86b954465bc1945b9733a2b0a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 125b34b86b954465bc1945b9733a2b0a, State: Initialized, Role: FOLLOWER
Sep 23 15:04:58 dist-test-slave-cnb0 krb5kdc[28944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175898, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.129@KRBTEST.COM for kudu/127.13.75.190@KRBTEST.COM
I20260923 15:04:58.919375 29604 consensus_queue.cc:260] T aceb8ae41b9342c0bcb31a8feb501b71 P 125b34b86b954465bc1945b9733a2b0a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "125b34b86b954465bc1945b9733a2b0a" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 39471 } } peers { permanent_uuid: "fd45cc8453994d0fabe2a44f19a74598" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 44675 } } peers { permanent_uuid: "4c6169b10f2e4152be4f90c81836aadb" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 38233 } }
I20260923 15:04:58.919636 29604 ts_tablet_manager.cc:1428] T aceb8ae41b9342c0bcb31a8feb501b71 P 125b34b86b954465bc1945b9733a2b0a: Time spent starting tablet: real 0.002s user 0.005s sys 0.000s
I20260923 15:04:58.924718 29714 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:58.907580 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:40941 (local address 127.13.75.129:40115)
0923 15:04:58.907833 (+ 253us) negotiation.cc:107] Waiting for socket to connect
0923 15:04:58.907882 (+ 49us) client_negotiation.cc:174] Beginning negotiation
0923 15:04:58.908457 (+ 575us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:04:58.911556 (+ 3099us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:04:58.911561 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:04:58.912031 (+ 470us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:04:58.912370 (+ 339us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:04:58.912378 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:04:58.914404 (+ 2026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:04:58.914405 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:04:58.915913 (+ 1508us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:04:58.915916 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:04:58.917028 (+ 1112us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:04:58.917030 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:04:58.917189 (+ 159us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:04:58.917822 (+ 633us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:04:58.917841 (+ 19us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:04:58.920138 (+ 2297us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:04:58.922797 (+ 2659us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:04:58.922807 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:04:58.922809 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:04:58.923175 (+ 366us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:04:58.923511 (+ 336us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:04:58.923512 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:04:58.923512 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:04:58.923652 (+ 140us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:04:58.924109 (+ 457us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:04:58.924116 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:04:58.924256 (+ 140us) client_negotiation.cc:769] Sending connection context
0923 15:04:58.924342 (+ 86us) client_negotiation.cc:240] Negotiation successful
0923 15:04:58.924510 (+ 168us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":201,"thread_start_us":154,"threads_started":1}
I20260923 15:04:58.925410 29715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:58.907688 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.129:40115 (local address 127.13.75.190:40941)
0923 15:04:58.911065 (+ 3377us) server_negotiation.cc:206] Beginning negotiation
0923 15:04:58.911076 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0923 15:04:58.911124 (+ 48us) server_negotiation.cc:407] Connection header received
0923 15:04:58.911272 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:04:58.911280 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:04:58.911339 (+ 59us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:04:58.912527 (+ 1188us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:04:58.912621 (+ 94us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:04:58.914261 (+ 1640us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:04:58.916046 (+ 1785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:04:58.916887 (+ 841us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:04:58.917515 (+ 628us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:04:58.920291 (+ 2776us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:04:58.920297 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:04:58.920297 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:04:58.920311 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:04:58.922664 (+ 2353us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:04:58.923289 (+ 625us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:04:58.923293 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:04:58.923293 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:04:58.923419 (+ 126us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:04:58.923729 (+ 310us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:04:58.923729 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:04:58.923730 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:04:58.924026 (+ 296us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:04:58.924988 (+ 962us) server_negotiation.cc:1035] Waiting for connection context
0923 15:04:58.925085 (+ 97us) server_negotiation.cc:299] Negotiation successful
0923 15:04:58.925225 (+ 140us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":3309,"thread_start_us":90,"threads_started":1}
I20260923 15:04:58.926119 29712 heartbeater.cc:347] Connected to a master server at 127.13.75.190:40941
I20260923 15:04:58.926220 29712 heartbeater.cc:464] Registering TS with master...
I20260923 15:04:58.926389 29712 heartbeater.cc:511] Master 127.13.75.190:40941 requested a full tablet report, sending...
I20260923 15:04:58.927117 29524 ts_manager.cc:194] Registered new tserver with Master: 125b34b86b954465bc1945b9733a2b0a (127.13.75.129:39471)
I20260923 15:04:58.928304 29524 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.75.129@KRBTEST.COM'} at 127.13.75.129:40115
I20260923 15:04:58.929236 29712 heartbeater.cc:503] Master 127.13.75.190:40941 was elected leader, sending a full tablet report...
Sep 23 15:04:58 dist-test-slave-cnb0 krb5kdc[28944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175898, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.190@KRBTEST.COM for kudu/127.13.75.129@KRBTEST.COM
I20260923 15:04:58.937549 29719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:58.927707 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.129:39471 (local address 127.0.0.1:37136)
0923 15:04:58.928005 (+ 298us) negotiation.cc:107] Waiting for socket to connect
0923 15:04:58.928043 (+ 38us) client_negotiation.cc:174] Beginning negotiation
0923 15:04:58.928105 (+ 62us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:04:58.928817 (+ 712us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:04:58.928821 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:04:58.929124 (+ 303us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:04:58.929178 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:04:58.929180 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:04:58.930433 (+ 1253us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:04:58.930434 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:04:58.931486 (+ 1052us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:04:58.931489 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:04:58.932051 (+ 562us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:04:58.932051 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:04:58.932129 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:04:58.932623 (+ 494us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:04:58.932628 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:04:58.934327 (+ 1699us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:04:58.936419 (+ 2092us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:04:58.936426 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:04:58.936428 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:04:58.936630 (+ 202us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:04:58.936863 (+ 233us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:04:58.936863 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:04:58.936864 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:04:58.936920 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:04:58.937244 (+ 324us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:04:58.937250 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:04:58.937293 (+ 43us) client_negotiation.cc:769] Sending connection context
0923 15:04:58.937356 (+ 63us) client_negotiation.cc:240] Negotiation successful
0923 15:04:58.937476 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":268,"thread_start_us":109,"threads_started":1}
I20260923 15:04:58.937603 29720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:58.927965 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37136 (local address 127.13.75.129:39471)
0923 15:04:58.928613 (+ 648us) server_negotiation.cc:206] Beginning negotiation
0923 15:04:58.928617 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 15:04:58.928629 (+ 12us) server_negotiation.cc:407] Connection header received
0923 15:04:58.928661 (+ 32us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:04:58.928663 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:04:58.928680 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:04:58.928757 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:04:58.929275 (+ 518us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:04:58.930346 (+ 1071us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:04:58.931579 (+ 1233us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:04:58.931968 (+ 389us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:04:58.932041 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:04:58.934412 (+ 2371us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:04:58.934420 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:04:58.934421 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:04:58.934428 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:04:58.936309 (+ 1881us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:04:58.936714 (+ 405us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:04:58.936717 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:04:58.936717 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:04:58.936791 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:04:58.936985 (+ 194us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:04:58.936986 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:04:58.936986 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:04:58.937176 (+ 190us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:04:58.937260 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0923 15:04:58.937411 (+ 151us) server_negotiation.cc:299] Negotiation successful
0923 15:04:58.937511 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":618,"thread_start_us":91,"threads_started":1}
I20260923 15:04:58.938071 29646 tablet_service.cc:1514] Processing DeleteTablet for tablet aceb8ae41b9342c0bcb31a8feb501b71 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 15:04:38 UTC) from {username='kudu', principal='kudu/127.13.75.190@KRBTEST.COM'} at 127.0.0.1:37136
I20260923 15:04:58.938628 29721 tablet_replica.cc:330] stopping tablet replica
I20260923 15:04:58.938781 29721 raft_consensus.cc:2229] T aceb8ae41b9342c0bcb31a8feb501b71 P 125b34b86b954465bc1945b9733a2b0a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 15:04:58.938871 29721 raft_consensus.cc:2258] T aceb8ae41b9342c0bcb31a8feb501b71 P 125b34b86b954465bc1945b9733a2b0a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 15:04:58.939242 29721 ts_tablet_manager.cc:1905] T aceb8ae41b9342c0bcb31a8feb501b71 P 125b34b86b954465bc1945b9733a2b0a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 15:04:58.941850 29721 ts_tablet_manager.cc:1918] T aceb8ae41b9342c0bcb31a8feb501b71 P 125b34b86b954465bc1945b9733a2b0a: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 15:04:58.941931 29721 log.cc:1196] T aceb8ae41b9342c0bcb31a8feb501b71 P 125b34b86b954465bc1945b9733a2b0a: Deleting WAL directory at /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/minicluster-data/ts-0/wal/wals/aceb8ae41b9342c0bcb31a8feb501b71
I20260923 15:04:58.942157 29721 ts_tablet_manager.cc:1939] T aceb8ae41b9342c0bcb31a8feb501b71 P 125b34b86b954465bc1945b9733a2b0a: Deleting consensus metadata
I20260923 15:04:58.942683 29510 catalog_manager.cc:4841] TS 125b34b86b954465bc1945b9733a2b0a (127.13.75.129:39471): tablet aceb8ae41b9342c0bcb31a8feb501b71 (table test-table [id=b04244703ae64caca1848868211a7acc]) successfully deleted
W20260923 15:04:59.005017 29716 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:04:59.005254 29716 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:04:59.005281 29716 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:04:59.006438 29716 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:04:59.006472 29716 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:04:59.006500 29716 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.75.130
I20260923 15:04:59.008028 29716 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.75.148:37659
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.130.keytab
--principal=kudu/127.13.75.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.75.130:44675
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.75.130
--webserver_port=46595
--tserver_master_addrs=127.13.75.190:40941
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29716
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.75.130
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:04:59.008237 29716 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:04:59.008431 29716 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:04:59.013819 29726 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:04:59.013782 29727 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:04:59.013800 29729 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:04:59.014139 29716 server_base.cc:1034] running on GCE node
I20260923 15:04:59.014349 29716 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:04:59.014648 29716 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:04:59.015805 29716 hybrid_clock.cc:648] HybridClock initialized: now 1790175899015792 us; error 29 us; skew 500 ppm
Sep 23 15:04:59 dist-test-slave-cnb0 krb5kdc[28944](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175899, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:04:59.018481 29716 init.cc:377] Logged in from keytab as kudu/127.13.75.130@KRBTEST.COM (short username kudu)
I20260923 15:04:59.018988 29716 webserver.cc:460] Webserver started at http://127.13.75.130:46595/ using document root <none> and password file <none>
I20260923 15:04:59.019240 29716 fs_manager.cc:359] Metadata directory not provided
I20260923 15:04:59.019301 29716 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:04:59.020743 29716 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 15:04:59.021485 29736 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:04:59.021677 29716 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:04:59.021811 29716 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/minicluster-data/ts-1/wal
uuid: "fd45cc8453994d0fabe2a44f19a74598"
format_stamp: "Formatted at 2026-09-23 15:04:38 on dist-test-slave-cnb0"
server_key: "b82ae8a1cf14878afae4c201907dbf64"
server_key_iv: "2e7ee59bb01f910981ccc6d03c7cb38e"
server_key_version: "encryptionkey@0"
I20260923 15:04:59.022190 29716 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:04:59.048032 29716 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:04:59.050176 29716 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:04:59.050370 29716 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:04:59.050635 29716 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 15:04:59.051047 29716 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 15:04:59.051107 29716 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 15:04:59.051152 29716 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 15:04:59.051183 29716 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 15:04:59.058494 29716 rpc_server.cc:307] RPC server started. Bound to: 127.13.75.130:44675
I20260923 15:04:59.058537 29849 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.75.130:44675 every 8 connection(s)
I20260923 15:04:59.058990 29716 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/minicluster-data/ts-1/data/info.pb
Sep 23 15:04:59 dist-test-slave-cnb0 krb5kdc[28944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175899, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.130@KRBTEST.COM for kudu/127.13.75.190@KRBTEST.COM
I20260923 15:04:59.067382 13614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKC4soy/build/release/bin/kudu as pid 29716
I20260923 15:04:59.067720 13614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKC4soy/build/release/bin/kudu
/tmp/dist-test-taskKC4soy/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.75.131:38233
--local_ip_for_outbound_sockets=127.13.75.131
--tserver_master_addrs=127.13.75.190:40941
--webserver_port=42631
--webserver_interface=127.13.75.131
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.131.keytab
--principal=kudu/127.13.75.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.75.148:37659
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260923 15:04:59.069185 29715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:59.059758 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.130:51207 (local address 127.13.75.190:40941)
0923 15:04:59.059831 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0923 15:04:59.059833 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:04:59.060073 (+ 240us) server_negotiation.cc:407] Connection header received
0923 15:04:59.060636 (+ 563us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:04:59.060638 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:04:59.060654 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:04:59.060704 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:04:59.061808 (+ 1104us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:04:59.062334 (+ 526us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:04:59.063464 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:04:59.063690 (+ 226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:04:59.063774 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:04:59.066084 (+ 2310us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:04:59.066085 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:04:59.066085 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:04:59.066096 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:04:59.067487 (+ 1391us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:04:59.068045 (+ 558us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:04:59.068046 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:04:59.068046 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:04:59.068101 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:04:59.068408 (+ 307us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:04:59.068409 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:04:59.068409 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:04:59.068543 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:04:59.068609 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0923 15:04:59.068952 (+ 343us) server_negotiation.cc:299] Negotiation successful
0923 15:04:59.069043 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":46}
I20260923 15:04:59.069432 29852 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:59.059739 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:40941 (local address 127.13.75.130:51207)
0923 15:04:59.059938 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0923 15:04:59.059974 (+ 36us) client_negotiation.cc:174] Beginning negotiation
0923 15:04:59.060501 (+ 527us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:04:59.060783 (+ 282us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:04:59.060786 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:04:59.061490 (+ 704us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:04:59.061690 (+ 200us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:04:59.061695 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:04:59.062441 (+ 746us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:04:59.062443 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:04:59.063303 (+ 860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:04:59.063305 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:04:59.063801 (+ 496us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:04:59.063802 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:04:59.063937 (+ 135us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:04:59.064519 (+ 582us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:04:59.064527 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:04:59.065954 (+ 1427us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:04:59.067595 (+ 1641us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:04:59.067602 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:04:59.067605 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:04:59.067966 (+ 361us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:04:59.068172 (+ 206us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:04:59.068173 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:04:59.068173 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:04:59.068319 (+ 146us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:04:59.068619 (+ 300us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:04:59.068625 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:04:59.068800 (+ 175us) client_negotiation.cc:769] Sending connection context
0923 15:04:59.068924 (+ 124us) client_negotiation.cc:240] Negotiation successful
0923 15:04:59.069114 (+ 190us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":103,"threads_started":1}
I20260923 15:04:59.070029 29850 heartbeater.cc:347] Connected to a master server at 127.13.75.190:40941
I20260923 15:04:59.070140 29850 heartbeater.cc:464] Registering TS with master...
I20260923 15:04:59.070364 29850 heartbeater.cc:511] Master 127.13.75.190:40941 requested a full tablet report, sending...
I20260923 15:04:59.070844 29524 ts_manager.cc:194] Registered new tserver with Master: fd45cc8453994d0fabe2a44f19a74598 (127.13.75.130:44675)
I20260923 15:04:59.071404 29524 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.75.130@KRBTEST.COM'} at 127.13.75.130:51207
W20260923 15:04:59.153216 29853 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:04:59.153435 29853 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:04:59.153460 29853 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:04:59.154623 29853 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:04:59.154662 29853 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:04:59.154690 29853 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.75.131
I20260923 15:04:59.156126 29853 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.75.148:37659
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.131.keytab
--principal=kudu/127.13.75.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.75.131:38233
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.75.131
--webserver_port=42631
--tserver_master_addrs=127.13.75.190:40941
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.29853
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.75.131
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:04:59.156325 29853 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:04:59.156507 29853 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:04:59.161515 29859 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:04:59.161533 29858 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:04:59.161522 29861 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:04:59.161767 29853 server_base.cc:1034] running on GCE node
I20260923 15:04:59.162010 29853 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:04:59.162268 29853 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:04:59.163434 29853 hybrid_clock.cc:648] HybridClock initialized: now 1790175899163430 us; error 42 us; skew 500 ppm
Sep 23 15:04:59 dist-test-slave-cnb0 krb5kdc[28944](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175899, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:04:59.165920 29853 init.cc:377] Logged in from keytab as kudu/127.13.75.131@KRBTEST.COM (short username kudu)
I20260923 15:04:59.166399 29853 webserver.cc:460] Webserver started at http://127.13.75.131:42631/ using document root <none> and password file <none>
I20260923 15:04:59.166622 29853 fs_manager.cc:359] Metadata directory not provided
I20260923 15:04:59.166702 29853 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:04:59.168143 29853 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:04:59.168751 29868 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:04:59.168910 29853 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 15:04:59.169006 29853 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/minicluster-data/ts-2/wal
uuid: "4c6169b10f2e4152be4f90c81836aadb"
format_stamp: "Formatted at 2026-09-23 15:04:38 on dist-test-slave-cnb0"
server_key: "0e829147c691b0c79bba8c0a6cab77a2"
server_key_iv: "930ca62486c0322d96007a120943a966"
server_key_version: "encryptionkey@0"
I20260923 15:04:59.169379 29853 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:04:59.177832 29853 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:04:59.179693 29853 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:04:59.179850 29853 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:04:59.180079 29853 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 15:04:59.180394 29853 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 15:04:59.180450 29853 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 15:04:59.180491 29853 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 15:04:59.180521 29853 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 15:04:59.187508 29853 rpc_server.cc:307] RPC server started. Bound to: 127.13.75.131:38233
I20260923 15:04:59.187557 29981 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.75.131:38233 every 8 connection(s)
I20260923 15:04:59.187918 29853 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/minicluster-data/ts-2/data/info.pb
I20260923 15:04:59.193794 13614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKC4soy/build/release/bin/kudu as pid 29853
Sep 23 15:04:59 dist-test-slave-cnb0 krb5kdc[28944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175899, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.131@KRBTEST.COM for kudu/127.13.75.190@KRBTEST.COM
I20260923 15:04:59.198292 29715 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:59.188702 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.131:50583 (local address 127.13.75.190:40941)
0923 15:04:59.188770 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0923 15:04:59.188772 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:04:59.188998 (+ 226us) server_negotiation.cc:407] Connection header received
0923 15:04:59.189537 (+ 539us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:04:59.189539 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:04:59.189551 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:04:59.189601 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:04:59.190540 (+ 939us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:04:59.191210 (+ 670us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:04:59.192263 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:04:59.192467 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:04:59.192541 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:04:59.194919 (+ 2378us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:04:59.194921 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:04:59.194921 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:04:59.194930 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:04:59.196672 (+ 1742us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:04:59.197126 (+ 454us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:04:59.197127 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:04:59.197128 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:04:59.197193 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:04:59.197475 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:04:59.197475 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:04:59.197476 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:04:59.197620 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:04:59.197672 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0923 15:04:59.198080 (+ 408us) server_negotiation.cc:299] Negotiation successful
0923 15:04:59.198174 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260923 15:04:59.198500 29984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:59.188708 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:40941 (local address 127.13.75.131:50583)
0923 15:04:59.188868 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0923 15:04:59.188909 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0923 15:04:59.189420 (+ 511us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:04:59.189674 (+ 254us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:04:59.189678 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:04:59.190244 (+ 566us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:04:59.190439 (+ 195us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:04:59.190444 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:04:59.191298 (+ 854us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:04:59.191299 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:04:59.192166 (+ 867us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:04:59.192169 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:04:59.192560 (+ 391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:04:59.192561 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:04:59.192683 (+ 122us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:04:59.193204 (+ 521us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:04:59.193212 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:04:59.194804 (+ 1592us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:04:59.196789 (+ 1985us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:04:59.196795 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:04:59.196797 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:04:59.197039 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:04:59.197282 (+ 243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:04:59.197283 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:04:59.197283 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:04:59.197401 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:04:59.197701 (+ 300us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:04:59.197709 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:04:59.197891 (+ 182us) client_negotiation.cc:769] Sending connection context
0923 15:04:59.198015 (+ 124us) client_negotiation.cc:240] Negotiation successful
0923 15:04:59.198196 (+ 181us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":63,"threads_started":1}
I20260923 15:04:59.199144 29982 heartbeater.cc:347] Connected to a master server at 127.13.75.190:40941
I20260923 15:04:59.199234 29982 heartbeater.cc:464] Registering TS with master...
I20260923 15:04:59.199433 29982 heartbeater.cc:511] Master 127.13.75.190:40941 requested a full tablet report, sending...
I20260923 15:04:59.199860 29524 ts_manager.cc:194] Registered new tserver with Master: 4c6169b10f2e4152be4f90c81836aadb (127.13.75.131:38233)
I20260923 15:04:59.200235 29524 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.75.131@KRBTEST.COM'} at 127.13.75.131:50583
I20260923 15:04:59.202425 29985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:59.194244 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55018 (local address 127.13.75.190:40941)
0923 15:04:59.194359 (+ 115us) server_negotiation.cc:206] Beginning negotiation
0923 15:04:59.194361 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:04:59.194594 (+ 233us) server_negotiation.cc:407] Connection header received
0923 15:04:59.194622 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:04:59.194623 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:04:59.194632 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:04:59.194693 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:04:59.195749 (+ 1056us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:04:59.196424 (+ 675us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:04:59.197551 (+ 1127us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:04:59.197834 (+ 283us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:04:59.197911 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:04:59.199741 (+ 1830us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:04:59.199743 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:04:59.199743 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:04:59.199750 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:04:59.201331 (+ 1581us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:04:59.201685 (+ 354us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:04:59.201685 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:04:59.201686 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:04:59.201732 (+ 46us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:04:59.201961 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:04:59.201961 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:04:59.201962 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:04:59.202056 (+ 94us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:04:59.202169 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0923 15:04:59.202298 (+ 129us) server_negotiation.cc:299] Negotiation successful
0923 15:04:59.202360 (+ 62us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":94,"thread_start_us":71,"threads_started":1}
I20260923 15:04:59.203296 13614 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260923 15:04:59.206007 29985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:59.203713 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55022 (local address 127.13.75.190:40941)
0923 15:04:59.203748 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0923 15:04:59.203750 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:04:59.203944 (+ 194us) server_negotiation.cc:407] Connection header received
0923 15:04:59.203979 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:04:59.203980 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:04:59.204008 (+ 28us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:04:59.204077 (+ 69us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 15:04:59.204197 (+ 120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:04:59.204655 (+ 458us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:04:59.205368 (+ 713us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:04:59.205576 (+ 208us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:04:59.205640 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:04:59.205758 (+ 118us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 15:04:59.205799 (+ 41us) server_negotiation.cc:394] Sending RPC error: FATAL_INVALID_AUTHENTICATION_TOKEN: Not authorized: token expired
0923 15:04:59.205922 (+ 123us) negotiation.cc:327] Negotiation complete: Not authorized: Server connection negotiation failed: server connection from 127.0.0.1:55022: token expired
Metrics: {"server-negotiator.queue_time_us":17}
W20260923 15:04:59.206070 29985 negotiation.cc:344] Unauthorized connection attempt: Server connection negotiation failed: server connection from 127.0.0.1:55022: token expired
W20260923 15:04:59.206171 29454 master_proxy_rpc.cc:203] Re-attempting CreateTable request to leader Master (127.13.75.190:40941)
I20260923 15:04:59.212395 29985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:59.206428 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:55024 (local address 127.13.75.190:40941)
0923 15:04:59.206473 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0923 15:04:59.206475 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:04:59.206485 (+ 10us) server_negotiation.cc:407] Connection header received
0923 15:04:59.206551 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:04:59.206552 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:04:59.206563 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:04:59.206622 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:04:59.207223 (+ 601us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:04:59.207678 (+ 455us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:04:59.208423 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:04:59.208625 (+ 202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:04:59.208719 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:04:59.209904 (+ 1185us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:04:59.209906 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:04:59.209906 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:04:59.209913 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:04:59.211221 (+ 1308us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:04:59.211576 (+ 355us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:04:59.211577 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:04:59.211577 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:04:59.211638 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:04:59.211856 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:04:59.211857 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:04:59.211857 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:04:59.211967 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:04:59.212075 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0923 15:04:59.212208 (+ 133us) server_negotiation.cc:299] Negotiation successful
0923 15:04:59.212292 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":22}
I20260923 15:04:59.213658 29524 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55024:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 15:04:59.215642 29524 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 15:04:59.218083 29646 tablet_service.cc:1467] Processing CreateTablet for tablet fa2002185f6a4877a2def06049cf198b (DEFAULT_TABLE table=test-table [id=99dc31466d704654a68228a718cdaa8c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 15:04:59.218381 29646 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fa2002185f6a4877a2def06049cf198b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:04:59.220199 29604 tablet_bootstrap.cc:492] T fa2002185f6a4877a2def06049cf198b P 125b34b86b954465bc1945b9733a2b0a: Bootstrap starting.
I20260923 15:04:59.220534 29604 tablet_bootstrap.cc:654] T fa2002185f6a4877a2def06049cf198b P 125b34b86b954465bc1945b9733a2b0a: Neither blocks nor log segments found. Creating new log.
I20260923 15:04:59.221125 29604 tablet_bootstrap.cc:492] T fa2002185f6a4877a2def06049cf198b P 125b34b86b954465bc1945b9733a2b0a: No bootstrap required, opened a new log
I20260923 15:04:59.221175 29604 ts_tablet_manager.cc:1397] T fa2002185f6a4877a2def06049cf198b P 125b34b86b954465bc1945b9733a2b0a: Time spent bootstrapping tablet: real 0.001s user 0.001s sys 0.000s
I20260923 15:04:59.221578 29604 raft_consensus.cc:348] T fa2002185f6a4877a2def06049cf198b P 125b34b86b954465bc1945b9733a2b0a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd45cc8453994d0fabe2a44f19a74598" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 44675 } } peers { permanent_uuid: "4c6169b10f2e4152be4f90c81836aadb" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 38233 } } peers { permanent_uuid: "125b34b86b954465bc1945b9733a2b0a" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 39471 } }
I20260923 15:04:59.221652 29604 raft_consensus.cc:374] T fa2002185f6a4877a2def06049cf198b P 125b34b86b954465bc1945b9733a2b0a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:04:59.221696 29604 raft_consensus.cc:729] T fa2002185f6a4877a2def06049cf198b P 125b34b86b954465bc1945b9733a2b0a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 125b34b86b954465bc1945b9733a2b0a, State: Initialized, Role: FOLLOWER
I20260923 15:04:59.221750 29604 consensus_queue.cc:260] T fa2002185f6a4877a2def06049cf198b P 125b34b86b954465bc1945b9733a2b0a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd45cc8453994d0fabe2a44f19a74598" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 44675 } } peers { permanent_uuid: "4c6169b10f2e4152be4f90c81836aadb" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 38233 } } peers { permanent_uuid: "125b34b86b954465bc1945b9733a2b0a" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 39471 } }
I20260923 15:04:59.221866 29604 ts_tablet_manager.cc:1428] T fa2002185f6a4877a2def06049cf198b P 125b34b86b954465bc1945b9733a2b0a: Time spent starting tablet: real 0.001s user 0.001s sys 0.000s
I20260923 15:04:59.221879 29719 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:59.217785 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.130:44675 (local address 127.0.0.1:51874)
0923 15:04:59.217864 (+ 79us) negotiation.cc:107] Waiting for socket to connect
0923 15:04:59.217876 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 15:04:59.217917 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:04:59.218360 (+ 443us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:04:59.218361 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:04:59.218363 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:04:59.218415 (+ 52us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:04:59.218417 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:04:59.219680 (+ 1263us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:04:59.219681 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:04:59.220654 (+ 973us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:04:59.220657 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:04:59.221571 (+ 914us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:04:59.221572 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:04:59.221643 (+ 71us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:04:59.221665 (+ 22us) client_negotiation.cc:769] Sending connection context
0923 15:04:59.221753 (+ 88us) client_negotiation.cc:240] Negotiation successful
0923 15:04:59.221800 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":61}
I20260923 15:04:59.221894 29989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:59.217925 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:51874 (local address 127.13.75.130:44675)
0923 15:04:59.218155 (+ 230us) server_negotiation.cc:206] Beginning negotiation
0923 15:04:59.218160 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 15:04:59.218173 (+ 13us) server_negotiation.cc:407] Connection header received
0923 15:04:59.218220 (+ 47us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:04:59.218222 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:04:59.218240 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:04:59.218333 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:04:59.218510 (+ 177us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:04:59.219595 (+ 1085us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:04:59.220776 (+ 1181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:04:59.221487 (+ 711us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:04:59.221573 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:04:59.221636 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0923 15:04:59.221773 (+ 137us) server_negotiation.cc:299] Negotiation successful
0923 15:04:59.221814 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":196,"thread_start_us":103,"threads_started":1}
I20260923 15:04:59.222752 29784 tablet_service.cc:1467] Processing CreateTablet for tablet fa2002185f6a4877a2def06049cf198b (DEFAULT_TABLE table=test-table [id=99dc31466d704654a68228a718cdaa8c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 15:04:59.223035 29988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:59.217909 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.131:38233 (local address 127.0.0.1:40218)
0923 15:04:59.218047 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0923 15:04:59.218057 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 15:04:59.218096 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:04:59.218562 (+ 466us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:04:59.218563 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:04:59.218564 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:04:59.218599 (+ 35us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:04:59.218601 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:04:59.220213 (+ 1612us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:04:59.220214 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:04:59.221688 (+ 1474us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:04:59.221704 (+ 16us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:04:59.222730 (+ 1026us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:04:59.222731 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:04:59.222797 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:04:59.222801 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 15:04:59.222924 (+ 123us) client_negotiation.cc:240] Negotiation successful
0923 15:04:59.222968 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":97,"threads_started":1}
I20260923 15:04:59.223107 29990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:59.218142 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40218 (local address 127.13.75.131:38233)
0923 15:04:59.218340 (+ 198us) server_negotiation.cc:206] Beginning negotiation
0923 15:04:59.218344 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 15:04:59.218354 (+ 10us) server_negotiation.cc:407] Connection header received
0923 15:04:59.218389 (+ 35us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:04:59.218391 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:04:59.218407 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:04:59.218479 (+ 72us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:04:59.218700 (+ 221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:04:59.220102 (+ 1402us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:04:59.221812 (+ 1710us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:04:59.222642 (+ 830us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:04:59.222715 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:04:59.222755 (+ 40us) server_negotiation.cc:1035] Waiting for connection context
0923 15:04:59.222951 (+ 196us) server_negotiation.cc:299] Negotiation successful
0923 15:04:59.222995 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158,"thread_start_us":75,"threads_started":1}
I20260923 15:04:59.223129 29784 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fa2002185f6a4877a2def06049cf198b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:04:59.225850 29916 tablet_service.cc:1467] Processing CreateTablet for tablet fa2002185f6a4877a2def06049cf198b (DEFAULT_TABLE table=test-table [id=99dc31466d704654a68228a718cdaa8c]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 15:04:59.226145 29992 tablet_bootstrap.cc:492] T fa2002185f6a4877a2def06049cf198b P fd45cc8453994d0fabe2a44f19a74598: Bootstrap starting.
I20260923 15:04:59.226282 29916 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet fa2002185f6a4877a2def06049cf198b. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:04:59.226873 29992 tablet_bootstrap.cc:654] T fa2002185f6a4877a2def06049cf198b P fd45cc8453994d0fabe2a44f19a74598: Neither blocks nor log segments found. Creating new log.
I20260923 15:04:59.227429 29992 log.cc:824] T fa2002185f6a4877a2def06049cf198b P fd45cc8453994d0fabe2a44f19a74598: Log is configured to *not* fsync() on all Append() calls
I20260923 15:04:59.228214 29992 tablet_bootstrap.cc:492] T fa2002185f6a4877a2def06049cf198b P fd45cc8453994d0fabe2a44f19a74598: No bootstrap required, opened a new log
I20260923 15:04:59.228302 29992 ts_tablet_manager.cc:1397] T fa2002185f6a4877a2def06049cf198b P fd45cc8453994d0fabe2a44f19a74598: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260923 15:04:59.229550 29994 tablet_bootstrap.cc:492] T fa2002185f6a4877a2def06049cf198b P 4c6169b10f2e4152be4f90c81836aadb: Bootstrap starting.
I20260923 15:04:59.230314 29992 raft_consensus.cc:348] T fa2002185f6a4877a2def06049cf198b P fd45cc8453994d0fabe2a44f19a74598 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd45cc8453994d0fabe2a44f19a74598" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 44675 } } peers { permanent_uuid: "4c6169b10f2e4152be4f90c81836aadb" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 38233 } } peers { permanent_uuid: "125b34b86b954465bc1945b9733a2b0a" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 39471 } }
I20260923 15:04:59.230558 29992 raft_consensus.cc:374] T fa2002185f6a4877a2def06049cf198b P fd45cc8453994d0fabe2a44f19a74598 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:04:59.230583 29994 tablet_bootstrap.cc:654] T fa2002185f6a4877a2def06049cf198b P 4c6169b10f2e4152be4f90c81836aadb: Neither blocks nor log segments found. Creating new log.
I20260923 15:04:59.230603 29992 raft_consensus.cc:729] T fa2002185f6a4877a2def06049cf198b P fd45cc8453994d0fabe2a44f19a74598 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fd45cc8453994d0fabe2a44f19a74598, State: Initialized, Role: FOLLOWER
I20260923 15:04:59.230736 29992 consensus_queue.cc:260] T fa2002185f6a4877a2def06049cf198b P fd45cc8453994d0fabe2a44f19a74598 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd45cc8453994d0fabe2a44f19a74598" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 44675 } } peers { permanent_uuid: "4c6169b10f2e4152be4f90c81836aadb" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 38233 } } peers { permanent_uuid: "125b34b86b954465bc1945b9733a2b0a" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 39471 } }
I20260923 15:04:59.231236 29850 heartbeater.cc:503] Master 127.13.75.190:40941 was elected leader, sending a full tablet report...
I20260923 15:04:59.231242 29992 ts_tablet_manager.cc:1428] T fa2002185f6a4877a2def06049cf198b P fd45cc8453994d0fabe2a44f19a74598: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260923 15:04:59.231398 29994 log.cc:824] T fa2002185f6a4877a2def06049cf198b P 4c6169b10f2e4152be4f90c81836aadb: Log is configured to *not* fsync() on all Append() calls
I20260923 15:04:59.232385 29994 tablet_bootstrap.cc:492] T fa2002185f6a4877a2def06049cf198b P 4c6169b10f2e4152be4f90c81836aadb: No bootstrap required, opened a new log
I20260923 15:04:59.232457 29994 ts_tablet_manager.cc:1397] T fa2002185f6a4877a2def06049cf198b P 4c6169b10f2e4152be4f90c81836aadb: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260923 15:04:59.233803 29994 raft_consensus.cc:348] T fa2002185f6a4877a2def06049cf198b P 4c6169b10f2e4152be4f90c81836aadb [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd45cc8453994d0fabe2a44f19a74598" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 44675 } } peers { permanent_uuid: "4c6169b10f2e4152be4f90c81836aadb" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 38233 } } peers { permanent_uuid: "125b34b86b954465bc1945b9733a2b0a" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 39471 } }
I20260923 15:04:59.233964 29994 raft_consensus.cc:374] T fa2002185f6a4877a2def06049cf198b P 4c6169b10f2e4152be4f90c81836aadb [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:04:59.234011 29994 raft_consensus.cc:729] T fa2002185f6a4877a2def06049cf198b P 4c6169b10f2e4152be4f90c81836aadb [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4c6169b10f2e4152be4f90c81836aadb, State: Initialized, Role: FOLLOWER
I20260923 15:04:59.234120 29994 consensus_queue.cc:260] T fa2002185f6a4877a2def06049cf198b P 4c6169b10f2e4152be4f90c81836aadb [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd45cc8453994d0fabe2a44f19a74598" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 44675 } } peers { permanent_uuid: "4c6169b10f2e4152be4f90c81836aadb" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 38233 } } peers { permanent_uuid: "125b34b86b954465bc1945b9733a2b0a" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 39471 } }
I20260923 15:04:59.234416 29994 ts_tablet_manager.cc:1428] T fa2002185f6a4877a2def06049cf198b P 4c6169b10f2e4152be4f90c81836aadb: Time spent starting tablet: real 0.002s user 0.000s sys 0.002s
I20260923 15:04:59.234449 29982 heartbeater.cc:503] Master 127.13.75.190:40941 was elected leader, sending a full tablet report...
W20260923 15:04:59.309613 29851 tablet.cc:2369] T fa2002185f6a4877a2def06049cf198b P fd45cc8453994d0fabe2a44f19a74598: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 15:04:59.332202 29997 raft_consensus.cc:482] T fa2002185f6a4877a2def06049cf198b P 4c6169b10f2e4152be4f90c81836aadb [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 15:04:59.332326 29997 raft_consensus.cc:504] T fa2002185f6a4877a2def06049cf198b P 4c6169b10f2e4152be4f90c81836aadb [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd45cc8453994d0fabe2a44f19a74598" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 44675 } } peers { permanent_uuid: "4c6169b10f2e4152be4f90c81836aadb" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 38233 } } peers { permanent_uuid: "125b34b86b954465bc1945b9733a2b0a" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 39471 } }
I20260923 15:04:59.332724 29997 leader_election.cc:288] T fa2002185f6a4877a2def06049cf198b P 4c6169b10f2e4152be4f90c81836aadb [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers fd45cc8453994d0fabe2a44f19a74598 (127.13.75.130:44675), 125b34b86b954465bc1945b9733a2b0a (127.13.75.129:39471)
I20260923 15:04:59.336683 29989 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:59.333128 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.131:47981 (local address 127.13.75.130:44675)
0923 15:04:59.333182 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0923 15:04:59.333184 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:04:59.333190 (+ 6us) server_negotiation.cc:407] Connection header received
0923 15:04:59.333218 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:04:59.333219 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:04:59.333228 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:04:59.333271 (+ 43us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:04:59.333521 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:04:59.334117 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:04:59.335717 (+ 1600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:04:59.336313 (+ 596us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:04:59.336398 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:04:59.336414 (+ 16us) server_negotiation.cc:1035] Waiting for connection context
0923 15:04:59.336569 (+ 155us) server_negotiation.cc:299] Negotiation successful
0923 15:04:59.336610 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260923 15:04:59.336704 29720 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:59.332978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.131:52121 (local address 127.13.75.129:39471)
0923 15:04:59.333029 (+ 51us) server_negotiation.cc:206] Beginning negotiation
0923 15:04:59.333031 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:04:59.333154 (+ 123us) server_negotiation.cc:407] Connection header received
0923 15:04:59.333262 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:04:59.333263 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:04:59.333289 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:04:59.333327 (+ 38us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:04:59.333545 (+ 218us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:04:59.334312 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:04:59.335717 (+ 1405us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:04:59.336324 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:04:59.336397 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:04:59.336481 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0923 15:04:59.336569 (+ 88us) server_negotiation.cc:299] Negotiation successful
0923 15:04:59.336611 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260923 15:04:59.336691 29984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:59.332880 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.130:44675 (local address 127.13.75.131:47981)
0923 15:04:59.332942 (+ 62us) negotiation.cc:107] Waiting for socket to connect
0923 15:04:59.332953 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 15:04:59.332994 (+ 41us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:04:59.333362 (+ 368us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:04:59.333363 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:04:59.333365 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:04:59.333435 (+ 70us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:04:59.333437 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:04:59.334202 (+ 765us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:04:59.334203 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:04:59.335602 (+ 1399us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:04:59.335605 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:04:59.336405 (+ 800us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:04:59.336406 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:04:59.336488 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:04:59.336490 (+ 2us) client_negotiation.cc:769] Sending connection context
0923 15:04:59.336546 (+ 56us) client_negotiation.cc:240] Negotiation successful
0923 15:04:59.336593 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":39,"mutex_wait_us":57}
I20260923 15:04:59.336701 29998 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:59.332930 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.129:39471 (local address 127.13.75.131:52121)
0923 15:04:59.333088 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0923 15:04:59.333099 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 15:04:59.333183 (+ 84us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:04:59.333389 (+ 206us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:04:59.333389 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:04:59.333391 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:04:59.333467 (+ 76us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:04:59.333469 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:04:59.334389 (+ 920us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:04:59.334390 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:04:59.335614 (+ 1224us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:04:59.335616 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:04:59.336405 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:04:59.336406 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:04:59.336483 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:04:59.336487 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 15:04:59.336546 (+ 59us) client_negotiation.cc:240] Negotiation successful
0923 15:04:59.336593 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"mutex_wait_us":14,"thread_start_us":103,"threads_started":1}
I20260923 15:04:59.337030 29666 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fa2002185f6a4877a2def06049cf198b" candidate_uuid: "4c6169b10f2e4152be4f90c81836aadb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "125b34b86b954465bc1945b9733a2b0a" is_pre_election: true
I20260923 15:04:59.337208 29666 raft_consensus.cc:2454] T fa2002185f6a4877a2def06049cf198b P 125b34b86b954465bc1945b9733a2b0a [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4c6169b10f2e4152be4f90c81836aadb in term 0.
I20260923 15:04:59.337226 29804 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fa2002185f6a4877a2def06049cf198b" candidate_uuid: "4c6169b10f2e4152be4f90c81836aadb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd45cc8453994d0fabe2a44f19a74598" is_pre_election: true
I20260923 15:04:59.337455 29871 leader_election.cc:302] T fa2002185f6a4877a2def06049cf198b P 4c6169b10f2e4152be4f90c81836aadb [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 125b34b86b954465bc1945b9733a2b0a, 4c6169b10f2e4152be4f90c81836aadb; no voters:
I20260923 15:04:59.337461 29804 raft_consensus.cc:2454] T fa2002185f6a4877a2def06049cf198b P fd45cc8453994d0fabe2a44f19a74598 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 4c6169b10f2e4152be4f90c81836aadb in term 0.
I20260923 15:04:59.337590 29997 raft_consensus.cc:2781] T fa2002185f6a4877a2def06049cf198b P 4c6169b10f2e4152be4f90c81836aadb [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 15:04:59.337674 29997 raft_consensus.cc:482] T fa2002185f6a4877a2def06049cf198b P 4c6169b10f2e4152be4f90c81836aadb [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 15:04:59.337700 29997 raft_consensus.cc:3037] T fa2002185f6a4877a2def06049cf198b P 4c6169b10f2e4152be4f90c81836aadb [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:04:59.338438 29997 raft_consensus.cc:504] T fa2002185f6a4877a2def06049cf198b P 4c6169b10f2e4152be4f90c81836aadb [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd45cc8453994d0fabe2a44f19a74598" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 44675 } } peers { permanent_uuid: "4c6169b10f2e4152be4f90c81836aadb" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 38233 } } peers { permanent_uuid: "125b34b86b954465bc1945b9733a2b0a" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 39471 } }
I20260923 15:04:59.338578 29997 leader_election.cc:288] T fa2002185f6a4877a2def06049cf198b P 4c6169b10f2e4152be4f90c81836aadb [CANDIDATE]: Term 1 election: Requested vote from peers fd45cc8453994d0fabe2a44f19a74598 (127.13.75.130:44675), 125b34b86b954465bc1945b9733a2b0a (127.13.75.129:39471)
I20260923 15:04:59.338757 29804 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fa2002185f6a4877a2def06049cf198b" candidate_uuid: "4c6169b10f2e4152be4f90c81836aadb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "fd45cc8453994d0fabe2a44f19a74598"
I20260923 15:04:59.338797 29666 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "fa2002185f6a4877a2def06049cf198b" candidate_uuid: "4c6169b10f2e4152be4f90c81836aadb" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "125b34b86b954465bc1945b9733a2b0a"
I20260923 15:04:59.338868 29666 raft_consensus.cc:3037] T fa2002185f6a4877a2def06049cf198b P 125b34b86b954465bc1945b9733a2b0a [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:04:59.338871 29804 raft_consensus.cc:3037] T fa2002185f6a4877a2def06049cf198b P fd45cc8453994d0fabe2a44f19a74598 [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:04:59.339958 29804 raft_consensus.cc:2454] T fa2002185f6a4877a2def06049cf198b P fd45cc8453994d0fabe2a44f19a74598 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4c6169b10f2e4152be4f90c81836aadb in term 1.
I20260923 15:04:59.339958 29666 raft_consensus.cc:2454] T fa2002185f6a4877a2def06049cf198b P 125b34b86b954465bc1945b9733a2b0a [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 4c6169b10f2e4152be4f90c81836aadb in term 1.
I20260923 15:04:59.340122 29871 leader_election.cc:302] T fa2002185f6a4877a2def06049cf198b P 4c6169b10f2e4152be4f90c81836aadb [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 125b34b86b954465bc1945b9733a2b0a, 4c6169b10f2e4152be4f90c81836aadb; no voters:
I20260923 15:04:59.340252 29997 raft_consensus.cc:2781] T fa2002185f6a4877a2def06049cf198b P 4c6169b10f2e4152be4f90c81836aadb [term 1 FOLLOWER]: Leader election won for term 1
I20260923 15:04:59.340456 29997 raft_consensus.cc:686] T fa2002185f6a4877a2def06049cf198b P 4c6169b10f2e4152be4f90c81836aadb [term 1 LEADER]: Becoming Leader. State: Replica: 4c6169b10f2e4152be4f90c81836aadb, State: Running, Role: LEADER
I20260923 15:04:59.340569 29997 consensus_queue.cc:237] T fa2002185f6a4877a2def06049cf198b P 4c6169b10f2e4152be4f90c81836aadb [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd45cc8453994d0fabe2a44f19a74598" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 44675 } } peers { permanent_uuid: "4c6169b10f2e4152be4f90c81836aadb" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 38233 } } peers { permanent_uuid: "125b34b86b954465bc1945b9733a2b0a" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 39471 } }
I20260923 15:04:59.341452 29524 catalog_manager.cc:5495] T fa2002185f6a4877a2def06049cf198b P 4c6169b10f2e4152be4f90c81836aadb reported cstate change: term changed from 0 to 1, leader changed from <none> to 4c6169b10f2e4152be4f90c81836aadb (127.13.75.131). New cstate: current_term: 1 leader_uuid: "4c6169b10f2e4152be4f90c81836aadb" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd45cc8453994d0fabe2a44f19a74598" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 44675 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "4c6169b10f2e4152be4f90c81836aadb" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 38233 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "125b34b86b954465bc1945b9733a2b0a" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 39471 } health_report { overall_health: UNKNOWN } } }
I20260923 15:04:59.367322 29990 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:59.364712 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40224 (local address 127.13.75.131:38233)
0923 15:04:59.364762 (+ 50us) server_negotiation.cc:206] Beginning negotiation
0923 15:04:59.364764 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:04:59.364773 (+ 9us) server_negotiation.cc:407] Connection header received
0923 15:04:59.364870 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:04:59.364871 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:04:59.364889 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:04:59.364960 (+ 71us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 15:04:59.365062 (+ 102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:04:59.365571 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:04:59.366377 (+ 806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:04:59.366596 (+ 219us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:04:59.366696 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:04:59.366818 (+ 122us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 15:04:59.366978 (+ 160us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 15:04:59.367048 (+ 70us) server_negotiation.cc:1035] Waiting for connection context
0923 15:04:59.367099 (+ 51us) server_negotiation.cc:299] Negotiation successful
0923 15:04:59.367119 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":25}
I20260923 15:04:59.369308 29666 raft_consensus.cc:1261] T fa2002185f6a4877a2def06049cf198b P 125b34b86b954465bc1945b9733a2b0a [term 1 FOLLOWER]: Refusing update from remote peer 4c6169b10f2e4152be4f90c81836aadb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 15:04:59.369366 29804 raft_consensus.cc:1261] T fa2002185f6a4877a2def06049cf198b P fd45cc8453994d0fabe2a44f19a74598 [term 1 FOLLOWER]: Refusing update from remote peer 4c6169b10f2e4152be4f90c81836aadb: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 15:04:59.369791 29997 consensus_queue.cc:1035] T fa2002185f6a4877a2def06049cf198b P 4c6169b10f2e4152be4f90c81836aadb [LEADER]: Connected to new peer: Peer: permanent_uuid: "125b34b86b954465bc1945b9733a2b0a" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 39471 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 15:04:59.369925 29999 consensus_queue.cc:1035] T fa2002185f6a4877a2def06049cf198b P 4c6169b10f2e4152be4f90c81836aadb [LEADER]: Connected to new peer: Peer: permanent_uuid: "fd45cc8453994d0fabe2a44f19a74598" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 44675 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 15:04:59.372417 30002 mvcc.cc:205] Tried to move back new op lower bound from 7332560483813703680 to 7332560483699699712. Current Snapshot: MvccSnapshot[applied={T|T < 7332560483813703680}]
I20260923 15:04:59.373607 30003 mvcc.cc:205] Tried to move back new op lower bound from 7332560483813703680 to 7332560483699699712. Current Snapshot: MvccSnapshot[applied={T|T < 7332560483813703680 or (T in {7332560483813703680})}]
I20260923 15:04:59.374612 29523 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55024:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 15:04:59.374744 29523 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:55024:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 15:04:59.375536 29523 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 842c3ea43bfc4cda9118b7f7d8dd212c: Sending DeleteTablet for 3 replicas of tablet fa2002185f6a4877a2def06049cf198b
I20260923 15:04:59.375893 29646 tablet_service.cc:1514] Processing DeleteTablet for tablet fa2002185f6a4877a2def06049cf198b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 15:04:59 UTC) from {username='kudu', principal='kudu/127.13.75.190@KRBTEST.COM'} at 127.0.0.1:37136
I20260923 15:04:59.375900 29784 tablet_service.cc:1514] Processing DeleteTablet for tablet fa2002185f6a4877a2def06049cf198b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 15:04:59 UTC) from {username='kudu', principal='kudu/127.13.75.190@KRBTEST.COM'} at 127.0.0.1:51874
I20260923 15:04:59.376036 29916 tablet_service.cc:1514] Processing DeleteTablet for tablet fa2002185f6a4877a2def06049cf198b with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 15:04:59 UTC) from {username='kudu', principal='kudu/127.13.75.190@KRBTEST.COM'} at 127.0.0.1:40218
I20260923 15:04:59.376214 29721 tablet_replica.cc:330] T fa2002185f6a4877a2def06049cf198b P 125b34b86b954465bc1945b9733a2b0a: stopping tablet replica
I20260923 15:04:59.376279 29721 raft_consensus.cc:2229] T fa2002185f6a4877a2def06049cf198b P 125b34b86b954465bc1945b9733a2b0a [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 15:04:59.376271 30011 tablet_replica.cc:330] T fa2002185f6a4877a2def06049cf198b P 4c6169b10f2e4152be4f90c81836aadb: stopping tablet replica
I20260923 15:04:59.376343 29721 raft_consensus.cc:2258] T fa2002185f6a4877a2def06049cf198b P 125b34b86b954465bc1945b9733a2b0a [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 15:04:59.376379 30011 raft_consensus.cc:2229] T fa2002185f6a4877a2def06049cf198b P 4c6169b10f2e4152be4f90c81836aadb [term 1 LEADER]: Raft consensus shutting down.
I20260923 15:04:59.376482 30011 raft_consensus.cc:2258] T fa2002185f6a4877a2def06049cf198b P 4c6169b10f2e4152be4f90c81836aadb [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 15:04:59.376636 13614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKC4soy/build/release/bin/kudu with pid 29562
I20260923 15:04:59.376677 29721 ts_tablet_manager.cc:1905] T fa2002185f6a4877a2def06049cf198b P 125b34b86b954465bc1945b9733a2b0a: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 15:04:59.377511 30011 ts_tablet_manager.cc:1905] T fa2002185f6a4877a2def06049cf198b P 4c6169b10f2e4152be4f90c81836aadb: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 15:04:59.379135 30011 ts_tablet_manager.cc:1918] T fa2002185f6a4877a2def06049cf198b P 4c6169b10f2e4152be4f90c81836aadb: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 15:04:59.379231 30011 log.cc:1196] T fa2002185f6a4877a2def06049cf198b P 4c6169b10f2e4152be4f90c81836aadb: Deleting WAL directory at /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/minicluster-data/ts-2/wal/wals/fa2002185f6a4877a2def06049cf198b
I20260923 15:04:59.379485 30011 ts_tablet_manager.cc:1939] T fa2002185f6a4877a2def06049cf198b P 4c6169b10f2e4152be4f90c81836aadb: Deleting consensus metadata
I20260923 15:04:59.379871 29511 catalog_manager.cc:4841] TS 4c6169b10f2e4152be4f90c81836aadb (127.13.75.131:38233): tablet fa2002185f6a4877a2def06049cf198b (table test-table [id=99dc31466d704654a68228a718cdaa8c]) successfully deleted
I20260923 15:04:59.383030 30010 tablet_replica.cc:330] T fa2002185f6a4877a2def06049cf198b P fd45cc8453994d0fabe2a44f19a74598: stopping tablet replica
I20260923 15:04:59.383173 30010 raft_consensus.cc:2229] T fa2002185f6a4877a2def06049cf198b P fd45cc8453994d0fabe2a44f19a74598 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 15:04:59.383344 30010 raft_consensus.cc:2258] T fa2002185f6a4877a2def06049cf198b P fd45cc8453994d0fabe2a44f19a74598 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 15:04:59.383781 30010 ts_tablet_manager.cc:1905] T fa2002185f6a4877a2def06049cf198b P fd45cc8453994d0fabe2a44f19a74598: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260923 15:04:59.384008 29510 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.75.129:39471 (error 108)
W20260923 15:04:59.384775 29988 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 15:04:59.384391 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.129:39471 (local address 127.0.0.1:37152)
0923 15:04:59.384452 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0923 15:04:59.384466 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0923 15:04:59.384519 (+ 53us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:04:59.384727 (+ 208us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.129:39471: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":39}
I20260923 15:04:59.384981 13614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKC4soy/build/release/bin/kudu with pid 29716
W20260923 15:04:59.385020 29510 catalog_manager.cc:4584] TS 125b34b86b954465bc1945b9733a2b0a (127.13.75.129:39471): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet fa2002185f6a4877a2def06049cf198b: Network error: Client connection negotiation failed: client connection to 127.13.75.129:39471: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260923 15:04:59.391314 13614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKC4soy/build/release/bin/kudu with pid 29853
I20260923 15:04:59.391456 29988 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:04:59.391276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.130:44675 (local address 127.0.0.1:51888)
0923 15:04:59.391329 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0923 15:04:59.391400 (+ 71us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.130:44675: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
I20260923 15:04:59.396041 13614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKC4soy/build/release/bin/kudu with pid 29491
I20260923 15:05:19.401614 13614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKC4soy/build/release/bin/kudu
/tmp/dist-test-taskKC4soy/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.75.190:40941
--webserver_interface=127.13.75.190
--webserver_port=35917
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.190.keytab
--principal=kudu/127.13.75.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.75.148:37659
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.75.190:40941
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20 with env {KRB5CCNAME=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:05:19.488471 30013 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:05:19.488682 30013 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:05:19.488705 30013 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:05:19.489796 30013 flags.cc:425] Enabled experimental flag: --authn_token_validity_seconds=20
W20260923 15:05:19.489833 30013 flags.cc:425] Enabled experimental flag: --authz_token_validity_seconds=20
W20260923 15:05:19.489846 30013 flags.cc:425] Enabled experimental flag: --tsk_rotation_seconds=20
W20260923 15:05:19.489858 30013 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 15:05:19.489871 30013 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:05:19.489884 30013 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:05:19.489897 30013 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 15:05:19.489909 30013 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 15:05:19.491433 30013 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.75.148:37659
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/minicluster-data/master-0/wal
--authn_token_validity_seconds=20
--authz_token_validity_seconds=20
--tsk_rotation_seconds=20
--ipki_ca_key_size=768
--master_addresses=127.13.75.190:40941
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.190.keytab
--principal=kudu/127.13.75.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.75.190:40941
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.75.190
--webserver_port=35917
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30013
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:05:19.491627 30013 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:05:19.491799 30013 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260923 15:05:19.497620 30013 server_base.cc:1034] running on GCE node
W20260923 15:05:19.497670 30019 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:05:19.497756 30021 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:05:19.497655 30018 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:05:19.498097 30013 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:05:19.498402 30013 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:05:19.499634 30013 hybrid_clock.cc:648] HybridClock initialized: now 1790175919499612 us; error 30 us; skew 500 ppm
Sep 23 15:05:19 dist-test-slave-cnb0 krb5kdc[28944](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175919, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:05:19.502627 30013 init.cc:377] Logged in from keytab as kudu/127.13.75.190@KRBTEST.COM (short username kudu)
I20260923 15:05:19.503225 30013 webserver.cc:460] Webserver started at http://127.13.75.190:35917/ using document root <none> and password file <none>
I20260923 15:05:19.503515 30013 fs_manager.cc:359] Metadata directory not provided
I20260923 15:05:19.503594 30013 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:05:19.505043 30013 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 15:05:19.505791 30028 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:05:19.505985 30013 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:05:19.506078 30013 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/minicluster-data/master-0/data,/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/minicluster-data/master-0/wal
uuid: "842c3ea43bfc4cda9118b7f7d8dd212c"
format_stamp: "Formatted at 2026-09-23 15:04:37 on dist-test-slave-cnb0"
server_key: "c5bd9a1448bd7566ed2d489c8e7943bb"
server_key_iv: "59b41eac661e9094ff8487caada62cc3"
server_key_version: "encryptionkey@0"
I20260923 15:05:19.506788 30013 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:05:19.524582 30013 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:05:19.526780 30013 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:05:19.527007 30013 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:05:19.531087 30013 rpc_server.cc:307] RPC server started. Bound to: 127.13.75.190:40941
I20260923 15:05:19.531481 30080 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.75.190:40941 every 8 connection(s)
I20260923 15:05:19.531523 30013 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/minicluster-data/master-0/data/info.pb
I20260923 15:05:19.532788 30081 sys_catalog.cc:263] Verifying existing consensus state
I20260923 15:05:19.533454 30081 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 842c3ea43bfc4cda9118b7f7d8dd212c: Bootstrap starting.
I20260923 15:05:19.537292 13614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKC4soy/build/release/bin/kudu as pid 30013
I20260923 15:05:19.537654 13614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKC4soy/build/release/bin/kudu
/tmp/dist-test-taskKC4soy/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.75.129:39471
--local_ip_for_outbound_sockets=127.13.75.129
--tserver_master_addrs=127.13.75.190:40941
--webserver_port=41567
--webserver_interface=127.13.75.129
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.129.keytab
--principal=kudu/127.13.75.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.75.148:37659
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260923 15:05:19.543426 30081 log.cc:824] T 00000000000000000000000000000000 P 842c3ea43bfc4cda9118b7f7d8dd212c: Log is configured to *not* fsync() on all Append() calls
I20260923 15:05:19.545763 30081 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 842c3ea43bfc4cda9118b7f7d8dd212c: Bootstrap replayed 1/1 log segments. Stats: ops{read=14 overwritten=0 applied=14 ignored=0} inserts{seen=8 ignored=0} mutations{seen=8 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260923 15:05:19.546093 30081 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 842c3ea43bfc4cda9118b7f7d8dd212c: Bootstrap complete.
I20260923 15:05:19.548506 30081 raft_consensus.cc:348] T 00000000000000000000000000000000 P 842c3ea43bfc4cda9118b7f7d8dd212c [term 2 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "842c3ea43bfc4cda9118b7f7d8dd212c" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 40941 } }
I20260923 15:05:19.548738 30081 raft_consensus.cc:729] T 00000000000000000000000000000000 P 842c3ea43bfc4cda9118b7f7d8dd212c [term 2 FOLLOWER]: Becoming Follower/Learner. State: Replica: 842c3ea43bfc4cda9118b7f7d8dd212c, State: Initialized, Role: FOLLOWER
I20260923 15:05:19.548908 30081 consensus_queue.cc:260] T 00000000000000000000000000000000 P 842c3ea43bfc4cda9118b7f7d8dd212c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "842c3ea43bfc4cda9118b7f7d8dd212c" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 40941 } }
I20260923 15:05:19.548978 30081 raft_consensus.cc:388] T 00000000000000000000000000000000 P 842c3ea43bfc4cda9118b7f7d8dd212c [term 2 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 15:05:19.549045 30081 raft_consensus.cc:482] T 00000000000000000000000000000000 P 842c3ea43bfc4cda9118b7f7d8dd212c [term 2 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 15:05:19.549089 30081 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 842c3ea43bfc4cda9118b7f7d8dd212c [term 2 FOLLOWER]: Advancing to term 3
I20260923 15:05:19.550105 30081 raft_consensus.cc:504] T 00000000000000000000000000000000 P 842c3ea43bfc4cda9118b7f7d8dd212c [term 3 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "842c3ea43bfc4cda9118b7f7d8dd212c" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 40941 } }
I20260923 15:05:19.550230 30081 leader_election.cc:302] T 00000000000000000000000000000000 P 842c3ea43bfc4cda9118b7f7d8dd212c [CANDIDATE]: Term 3 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 842c3ea43bfc4cda9118b7f7d8dd212c; no voters:
I20260923 15:05:19.550453 30081 leader_election.cc:288] T 00000000000000000000000000000000 P 842c3ea43bfc4cda9118b7f7d8dd212c [CANDIDATE]: Term 3 election: Requested vote from peers
I20260923 15:05:19.550493 30085 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 842c3ea43bfc4cda9118b7f7d8dd212c [term 3 FOLLOWER]: Leader election won for term 3
I20260923 15:05:19.550616 30085 raft_consensus.cc:686] T 00000000000000000000000000000000 P 842c3ea43bfc4cda9118b7f7d8dd212c [term 3 LEADER]: Becoming Leader. State: Replica: 842c3ea43bfc4cda9118b7f7d8dd212c, State: Running, Role: LEADER
I20260923 15:05:19.550719 30085 consensus_queue.cc:237] T 00000000000000000000000000000000 P 842c3ea43bfc4cda9118b7f7d8dd212c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 14, Committed index: 14, Last appended: 2.14, Last appended by leader: 14, Current term: 3, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "842c3ea43bfc4cda9118b7f7d8dd212c" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 40941 } }
I20260923 15:05:19.550724 30081 sys_catalog.cc:564] T 00000000000000000000000000000000 P 842c3ea43bfc4cda9118b7f7d8dd212c [sys.catalog]: configured and running, proceeding with master startup.
I20260923 15:05:19.551077 30086 sys_catalog.cc:455] T 00000000000000000000000000000000 P 842c3ea43bfc4cda9118b7f7d8dd212c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 3 leader_uuid: "842c3ea43bfc4cda9118b7f7d8dd212c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "842c3ea43bfc4cda9118b7f7d8dd212c" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 40941 } } }
I20260923 15:05:19.551177 30086 sys_catalog.cc:458] T 00000000000000000000000000000000 P 842c3ea43bfc4cda9118b7f7d8dd212c [sys.catalog]: This master's current role is: LEADER
I20260923 15:05:19.551137 30087 sys_catalog.cc:455] T 00000000000000000000000000000000 P 842c3ea43bfc4cda9118b7f7d8dd212c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 842c3ea43bfc4cda9118b7f7d8dd212c. Latest consensus state: current_term: 3 leader_uuid: "842c3ea43bfc4cda9118b7f7d8dd212c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "842c3ea43bfc4cda9118b7f7d8dd212c" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 40941 } } }
I20260923 15:05:19.551436 30087 sys_catalog.cc:458] T 00000000000000000000000000000000 P 842c3ea43bfc4cda9118b7f7d8dd212c [sys.catalog]: This master's current role is: LEADER
I20260923 15:05:19.551813 30089 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 15:05:19.552678 30089 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 15:05:19.552788 30089 catalog_manager.cc:1259] Loaded cluster ID: 5e9ad76930f74565ae7880afecf8f882
I20260923 15:05:19.552827 30089 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 15:05:19.553736 30089 catalog_manager.cc:1504] Loading token signing keys...
I20260923 15:05:19.553936 30089 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 842c3ea43bfc4cda9118b7f7d8dd212c: Loaded TSK: 0
I20260923 15:05:19.553995 30089 catalog_manager.cc:5879] T 00000000000000000000000000000000 P 842c3ea43bfc4cda9118b7f7d8dd212c: Loaded TSK: 1
F20260923 15:05:19.554221 30089 token_signing_key.cc:66] Check failed: _s.ok() Bad status: Runtime error: unable to load data from memory: error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=RSA error:04093004:rsa routines:OLD_RSA_PRIV_DECODE:RSA lib:rsa_ameth.c:121 error:0D0680A8:asn1 encoding routines:ASN1_CHECK_TLEN:wrong tag:tasn_dec.c:1220 error:0D06C03A:asn1 encoding routines:ASN1_D2I_EX_PRIMITIVE:nested asn1 error:tasn_dec.c:788 error:0D08303A:asn1 encoding routines:ASN1_TEMPLATE_NOEXP_D2I:nested asn1 error:tasn_dec.c:720:Field=version, Type=PKCS8_PRIV_KEY_INFO
*** Check failure stack trace: ***
@ 0x7f0bc9f6f9ad google::LogMessage::Fail() at ??:0
@ 0x7f0bc9f7288a google::LogMessage::SendToLog() at ??:0
@ 0x7f0bc9f6f4de google::LogMessage::Flush() at ??:0
@ 0x7f0bc9f70a19 google::LogMessageFatal::~LogMessageFatal() at ??:0
@ 0x7f0bcc6dc104 kudu::security::TokenSigningPrivateKey::TokenSigningPrivateKey() at ??:0
@ 0x7f0bcc6d9d41 kudu::security::TokenSigner::ImportKeys() at ??:0
@ 0x7f0bd1ec91cd kudu::master::CatalogManager::LoadTskEntries() at ??:0
@ 0x7f0bd1ec948e kudu::master::CatalogManager::InitTokenSigner() at ??:0
@ 0x7f0bd1ec9594 _ZNSt17_Function_handlerIFN4kudu6StatusEvEZNS0_6master14CatalogManager24PrepareForLeadershipTaskEvEUlvE3_E9_M_invokeERKSt9_Any_data at ??:0
@ 0x7f0bd1e8fc48 _ZZN4kudu6master14CatalogManager24PrepareForLeadershipTaskEvENKUlSt8functionIFNS_6StatusEvEERKNS_9consensus13RaftConsensusElPKcE_clES5_S9_lSB_.isra.591 at ??:0
@ 0x7f0bd1e99ad0 kudu::master::CatalogManager::PrepareForLeadershipTask() at ??:0
@ 0x7f0bcbe3d0d3 kudu::ThreadPool::DispatchThread() at ??:0
@ 0x7f0bcbe32954 kudu::Thread::SuperviseThread() at ??:0
@ 0x7f0bccc966db start_thread at ??:0
@ 0x7f0bc8a6d61f clone at ??:0
W20260923 15:05:19.641604 30083 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:05:19.641804 30083 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:05:19.641829 30083 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:05:19.643056 30083 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:05:19.643102 30083 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:05:19.643131 30083 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.75.129
I20260923 15:05:19.644640 30083 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.75.148:37659
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.129.keytab
--principal=kudu/127.13.75.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.75.129:39471
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.75.129
--webserver_port=41567
--tserver_master_addrs=127.13.75.190:40941
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30083
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.75.129
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:05:19.644918 30083 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:05:19.645161 30083 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:05:19.650367 30122 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:05:19.650367 30123 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:05:19.650414 30125 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:05:19.650746 30083 server_base.cc:1034] running on GCE node
I20260923 15:05:19.650982 30083 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:05:19.651238 30083 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:05:19.652441 30083 hybrid_clock.cc:648] HybridClock initialized: now 1790175919652423 us; error 30 us; skew 500 ppm
Sep 23 15:05:19 dist-test-slave-cnb0 krb5kdc[28944](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175919, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:05:19.655045 30083 init.cc:377] Logged in from keytab as kudu/127.13.75.129@KRBTEST.COM (short username kudu)
I20260923 15:05:19.655536 30083 webserver.cc:460] Webserver started at http://127.13.75.129:41567/ using document root <none> and password file <none>
I20260923 15:05:19.655784 30083 fs_manager.cc:359] Metadata directory not provided
I20260923 15:05:19.655831 30083 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:05:19.657197 30083 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:05:19.658028 30132 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:05:19.658226 30083 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:05:19.658326 30083 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/minicluster-data/ts-0/wal
uuid: "125b34b86b954465bc1945b9733a2b0a"
format_stamp: "Formatted at 2026-09-23 15:04:38 on dist-test-slave-cnb0"
server_key: "30fb4b9008d66fe51a6d9aab609f2492"
server_key_iv: "41a52838ed269bdd2774e65fb947e496"
server_key_version: "encryptionkey@0"
I20260923 15:05:19.658679 30083 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:05:19.670861 30083 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:05:19.673204 30083 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:05:19.673360 30083 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:05:19.673652 30083 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 15:05:19.674155 30139 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
I20260923 15:05:19.676131 30083 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 1 live tablets)
I20260923 15:05:19.676199 30083 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.002s user 0.000s sys 0.000s
I20260923 15:05:19.676245 30083 ts_tablet_manager.cc:594] Registering tablets (0/1 complete)
I20260923 15:05:19.676878 30083 ts_tablet_manager.cc:610] Registered 1 tablets
I20260923 15:05:19.676936 30083 ts_tablet_manager.cc:589] Time spent register tablets: real 0.001s user 0.000s sys 0.000s
I20260923 15:05:19.677007 30139 tablet_bootstrap.cc:492] T fa2002185f6a4877a2def06049cf198b P 125b34b86b954465bc1945b9733a2b0a: Bootstrap starting.
I20260923 15:05:19.678853 30139 log.cc:824] T fa2002185f6a4877a2def06049cf198b P 125b34b86b954465bc1945b9733a2b0a: Log is configured to *not* fsync() on all Append() calls
I20260923 15:05:19.681943 30139 tablet_bootstrap.cc:492] T fa2002185f6a4877a2def06049cf198b P 125b34b86b954465bc1945b9733a2b0a: Bootstrap replayed 1/1 log segments. Stats: ops{read=2 overwritten=0 applied=2 ignored=0} inserts{seen=1 ignored=0} mutations{seen=0 ignored=0} orphaned_commits=0. Pending: 0 replicates
I20260923 15:05:19.682294 30139 tablet_bootstrap.cc:492] T fa2002185f6a4877a2def06049cf198b P 125b34b86b954465bc1945b9733a2b0a: Bootstrap complete.
I20260923 15:05:19.682467 30139 ts_tablet_manager.cc:1397] T fa2002185f6a4877a2def06049cf198b P 125b34b86b954465bc1945b9733a2b0a: Time spent bootstrapping tablet: real 0.006s user 0.002s sys 0.000s
I20260923 15:05:19.684141 30083 rpc_server.cc:307] RPC server started. Bound to: 127.13.75.129:39471
I20260923 15:05:19.684477 30083 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/minicluster-data/ts-0/data/info.pb
I20260923 15:05:19.684430 30139 raft_consensus.cc:348] T fa2002185f6a4877a2def06049cf198b P 125b34b86b954465bc1945b9733a2b0a [term 1 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd45cc8453994d0fabe2a44f19a74598" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 44675 } } peers { permanent_uuid: "4c6169b10f2e4152be4f90c81836aadb" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 38233 } } peers { permanent_uuid: "125b34b86b954465bc1945b9733a2b0a" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 39471 } }
I20260923 15:05:19.684962 30139 raft_consensus.cc:729] T fa2002185f6a4877a2def06049cf198b P 125b34b86b954465bc1945b9733a2b0a [term 1 FOLLOWER]: Becoming Follower/Learner. State: Replica: 125b34b86b954465bc1945b9733a2b0a, State: Initialized, Role: FOLLOWER
I20260923 15:05:19.685073 30247 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.75.129:39471 every 8 connection(s)
I20260923 15:05:19.685132 30139 consensus_queue.cc:260] T fa2002185f6a4877a2def06049cf198b P 125b34b86b954465bc1945b9733a2b0a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 2, Last appended: 1.2, Last appended by leader: 2, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd45cc8453994d0fabe2a44f19a74598" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 44675 } } peers { permanent_uuid: "4c6169b10f2e4152be4f90c81836aadb" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 38233 } } peers { permanent_uuid: "125b34b86b954465bc1945b9733a2b0a" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 39471 } }
I20260923 15:05:19.685410 30139 ts_tablet_manager.cc:1428] T fa2002185f6a4877a2def06049cf198b P 125b34b86b954465bc1945b9733a2b0a: Time spent starting tablet: real 0.003s user 0.002s sys 0.003s
I20260923 15:05:19.694065 13614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKC4soy/build/release/bin/kudu as pid 30083
I20260923 15:05:19.694387 13614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKC4soy/build/release/bin/kudu
/tmp/dist-test-taskKC4soy/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.75.130:44675
--local_ip_for_outbound_sockets=127.13.75.130
--tserver_master_addrs=127.13.75.190:40941
--webserver_port=46595
--webserver_interface=127.13.75.130
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.130.keytab
--principal=kudu/127.13.75.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.75.148:37659
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:05:19.748826 30251 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 15:05:19.685202 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:40941 (local address 127.13.75.129:33703)
0923 15:05:19.685393 (+ 191us) negotiation.cc:107] Waiting for socket to connect
0923 15:05:19.685438 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0923 15:05:19.685588 (+ 150us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:05:19.748578 (+ 62990us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.190:40941: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":121,"threads_started":1}
W20260923 15:05:19.749085 30248 heartbeater.cc:644] Failed to heartbeat to 127.13.75.190:40941 (0 consecutive failures): Network error: Failed to ping master at 127.13.75.190:40941: Client connection negotiation failed: client connection to 127.13.75.190:40941: BlockingRecv error: recv error from unknown peer: Transport endpoint is not connected (error 107)
I20260923 15:05:19.749490 30251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:19.749370 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:40941 (local address 127.13.75.129:51219)
0923 15:05:19.749427 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0923 15:05:19.749469 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.190:40941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
I20260923 15:05:19.749851 30251 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:19.749754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:40941 (local address 127.13.75.129:39549)
0923 15:05:19.749801 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0923 15:05:19.749827 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.190:40941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260923 15:05:19.749961 30248 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
W20260923 15:05:19.779241 30252 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:05:19.779482 30252 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:05:19.779525 30252 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:05:19.780741 30252 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:05:19.780814 30252 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:05:19.780865 30252 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.75.130
I20260923 15:05:19.782423 30252 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.75.148:37659
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.130.keytab
--principal=kudu/127.13.75.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.75.130:44675
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.75.130
--webserver_port=46595
--tserver_master_addrs=127.13.75.190:40941
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30252
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.75.130
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:05:19.782693 30252 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:05:19.782984 30252 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:05:19.789038 30258 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:05:19.789032 30257 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:05:19.789032 30260 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:05:19.789269 30252 server_base.cc:1034] running on GCE node
I20260923 15:05:19.789604 30252 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:05:19.789824 30252 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:05:19.790979 30252 hybrid_clock.cc:648] HybridClock initialized: now 1790175919790955 us; error 28 us; skew 500 ppm
Sep 23 15:05:19 dist-test-slave-cnb0 krb5kdc[28944](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175919, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:05:19.793867 30252 init.cc:377] Logged in from keytab as kudu/127.13.75.130@KRBTEST.COM (short username kudu)
I20260923 15:05:19.794380 30252 webserver.cc:460] Webserver started at http://127.13.75.130:46595/ using document root <none> and password file <none>
I20260923 15:05:19.794636 30252 fs_manager.cc:359] Metadata directory not provided
I20260923 15:05:19.794682 30252 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:05:19.796126 30252 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:05:19.796875 30267 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:05:19.797097 30252 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:05:19.797169 30252 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/minicluster-data/ts-1/wal
uuid: "fd45cc8453994d0fabe2a44f19a74598"
format_stamp: "Formatted at 2026-09-23 15:04:38 on dist-test-slave-cnb0"
server_key: "b82ae8a1cf14878afae4c201907dbf64"
server_key_iv: "2e7ee59bb01f910981ccc6d03c7cb38e"
server_key_version: "encryptionkey@0"
I20260923 15:05:19.797600 30252 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:05:19.808856 30252 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:05:19.811015 30252 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:05:19.811165 30252 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:05:19.811420 30252 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 15:05:19.811916 30274 ts_tablet_manager.cc:536] Loading tablet metadata (0/1 complete)
W20260923 15:05:19.813517 30274 ts_tablet_manager.cc:1852] T fa2002185f6a4877a2def06049cf198b P fd45cc8453994d0fabe2a44f19a74598: Tablet Manager startup: Rolling forward tablet deletion of type TABLET_DATA_DELETED
I20260923 15:05:19.813812 30274 ts_tablet_manager.cc:1905] T fa2002185f6a4877a2def06049cf198b P fd45cc8453994d0fabe2a44f19a74598: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 15:05:19.816318 30274 ts_tablet_manager.cc:1918] T fa2002185f6a4877a2def06049cf198b P fd45cc8453994d0fabe2a44f19a74598: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 15:05:19.816431 30274 log.cc:1196] T fa2002185f6a4877a2def06049cf198b P fd45cc8453994d0fabe2a44f19a74598: Deleting WAL directory at /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/minicluster-data/ts-1/wal/wals/fa2002185f6a4877a2def06049cf198b
I20260923 15:05:19.816792 30274 ts_tablet_manager.cc:1939] T fa2002185f6a4877a2def06049cf198b P fd45cc8453994d0fabe2a44f19a74598: Deleting consensus metadata
I20260923 15:05:19.817045 30252 ts_tablet_manager.cc:579] Loaded tablet metadata (1 total tablets, 0 live tablets)
I20260923 15:05:19.817126 30252 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.005s user 0.000s sys 0.000s
I20260923 15:05:19.817171 30252 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 15:05:19.817198 30252 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 15:05:19.823547 30252 rpc_server.cc:307] RPC server started. Bound to: 127.13.75.130:44675
I20260923 15:05:19.823602 30381 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.75.130:44675 every 8 connection(s)
I20260923 15:05:19.823911 30252 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/minicluster-data/ts-1/data/info.pb
I20260923 15:05:19.824954 30384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:19.824576 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:40941 (local address 127.13.75.130:42203)
0923 15:05:19.824742 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0923 15:05:19.824819 (+ 77us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.190:40941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":65,"threads_started":1}
W20260923 15:05:19.825196 30382 heartbeater.cc:644] Failed to heartbeat to 127.13.75.190:40941 (0 consecutive failures): Network error: Failed to ping master at 127.13.75.190:40941: Client connection negotiation failed: client connection to 127.13.75.190:40941: connect: Connection refused (error 111)
I20260923 15:05:19.825486 30384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:19.825396 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:40941 (local address 127.13.75.130:34155)
0923 15:05:19.825441 (+ 45us) negotiation.cc:107] Waiting for socket to connect
0923 15:05:19.825467 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.190:40941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
I20260923 15:05:19.825837 30384 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:19.825692 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:40941 (local address 127.13.75.130:56853)
0923 15:05:19.825784 (+ 92us) negotiation.cc:107] Waiting for socket to connect
0923 15:05:19.825815 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.190:40941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":73}
W20260923 15:05:19.825977 30382 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260923 15:05:19.829825 13614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKC4soy/build/release/bin/kudu as pid 30252
I20260923 15:05:19.830142 13614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKC4soy/build/release/bin/kudu
/tmp/dist-test-taskKC4soy/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.75.131:38233
--local_ip_for_outbound_sockets=127.13.75.131
--tserver_master_addrs=127.13.75.190:40941
--webserver_port=42631
--webserver_interface=127.13.75.131
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.131.keytab
--principal=kudu/127.13.75.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.75.148:37659
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:05:19.911418 30385 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:05:19.911617 30385 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:05:19.911641 30385 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:05:19.912714 30385 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:05:19.912746 30385 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:05:19.912771 30385 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.75.131
I20260923 15:05:19.914121 30385 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.75.148:37659
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.131.keytab
--principal=kudu/127.13.75.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.75.131:38233
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.75.131
--webserver_port=42631
--tserver_master_addrs=127.13.75.190:40941
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30385
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.75.131
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:05:19.914304 30385 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:05:19.914539 30385 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:05:19.919679 30391 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:05:19.919636 30390 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:05:19.919831 30385 server_base.cc:1034] running on GCE node
W20260923 15:05:19.919636 30393 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:05:19.920185 30385 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:05:19.920502 30385 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:05:19.921672 30385 hybrid_clock.cc:648] HybridClock initialized: now 1790175919921655 us; error 34 us; skew 500 ppm
Sep 23 15:05:19 dist-test-slave-cnb0 krb5kdc[28944](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175919, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:05:19.924273 30385 init.cc:377] Logged in from keytab as kudu/127.13.75.131@KRBTEST.COM (short username kudu)
I20260923 15:05:19.924824 30385 webserver.cc:460] Webserver started at http://127.13.75.131:42631/ using document root <none> and password file <none>
I20260923 15:05:19.925072 30385 fs_manager.cc:359] Metadata directory not provided
I20260923 15:05:19.925191 30385 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:05:19.926612 30385 fs_manager.cc:709] Time spent opening directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 15:05:19.927377 30400 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:05:19.927554 30385 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260923 15:05:19.927651 30385 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/minicluster-data/ts-2/wal
uuid: "4c6169b10f2e4152be4f90c81836aadb"
format_stamp: "Formatted at 2026-09-23 15:04:38 on dist-test-slave-cnb0"
server_key: "0e829147c691b0c79bba8c0a6cab77a2"
server_key_iv: "930ca62486c0322d96007a120943a966"
server_key_version: "encryptionkey@0"
I20260923 15:05:19.928027 30385 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:05:19.941862 30385 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:05:19.944212 30385 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:05:19.944382 30385 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:05:19.944634 30385 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 15:05:19.944993 30385 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 15:05:19.945048 30385 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 15:05:19.945088 30385 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 15:05:19.945117 30385 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 15:05:19.951246 30385 rpc_server.cc:307] RPC server started. Bound to: 127.13.75.131:38233
I20260923 15:05:19.951289 30513 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.75.131:38233 every 8 connection(s)
I20260923 15:05:19.951666 30385 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.SecurityITest.TestEnableAndDisableEncryptedTSK.1790175691491353-13614-0/minicluster-data/ts-2/data/info.pb
I20260923 15:05:19.952772 30516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:19.952356 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:40941 (local address 127.13.75.131:58621)
0923 15:05:19.952538 (+ 182us) negotiation.cc:107] Waiting for socket to connect
0923 15:05:19.952617 (+ 79us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.190:40941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":70,"threads_started":1}
W20260923 15:05:19.953263 30514 heartbeater.cc:644] Failed to heartbeat to 127.13.75.190:40941 (0 consecutive failures): Network error: Failed to ping master at 127.13.75.190:40941: Client connection negotiation failed: client connection to 127.13.75.190:40941: connect: Connection refused (error 111)
I20260923 15:05:19.953599 30516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:19.953469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:40941 (local address 127.13.75.131:44293)
0923 15:05:19.953542 (+ 73us) negotiation.cc:107] Waiting for socket to connect
0923 15:05:19.953575 (+ 33us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.190:40941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":51}
I20260923 15:05:19.954032 30516 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:19.953904 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:40941 (local address 127.13.75.131:33295)
0923 15:05:19.953965 (+ 61us) negotiation.cc:107] Waiting for socket to connect
0923 15:05:19.954012 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.190:40941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260923 15:05:19.954229 30514 heartbeater.cc:415] Failed 3 heartbeats in a row: no longer allowing fast heartbeat attempts.
I20260923 15:05:19.955118 13614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKC4soy/build/release/bin/kudu as pid 30385
I20260923 15:05:20.750708 30518 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:20.750440 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:40941 (local address 127.13.75.129:47691)
0923 15:05:20.750589 (+ 149us) negotiation.cc:107] Waiting for socket to connect
0923 15:05:20.750625 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.190:40941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":132,"thread_start_us":77,"threads_started":1}
I20260923 15:05:20.826579 30519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:20.826350 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:40941 (local address 127.13.75.130:59517)
0923 15:05:20.826473 (+ 123us) negotiation.cc:107] Waiting for socket to connect
0923 15:05:20.826536 (+ 63us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.190:40941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":106,"thread_start_us":55,"threads_started":1}
I20260923 15:05:20.954775 30520 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:20.954541 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:40941 (local address 127.13.75.131:56755)
0923 15:05:20.954684 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0923 15:05:20.954722 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.190:40941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":55,"threads_started":1}
I20260923 15:05:21.483326 30521 raft_consensus.cc:482] T fa2002185f6a4877a2def06049cf198b P 125b34b86b954465bc1945b9733a2b0a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 15:05:21.483457 30521 raft_consensus.cc:504] T fa2002185f6a4877a2def06049cf198b P 125b34b86b954465bc1945b9733a2b0a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd45cc8453994d0fabe2a44f19a74598" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 44675 } } peers { permanent_uuid: "4c6169b10f2e4152be4f90c81836aadb" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 38233 } } peers { permanent_uuid: "125b34b86b954465bc1945b9733a2b0a" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 39471 } }
I20260923 15:05:21.483752 30521 leader_election.cc:288] T fa2002185f6a4877a2def06049cf198b P 125b34b86b954465bc1945b9733a2b0a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fd45cc8453994d0fabe2a44f19a74598 (127.13.75.130:44675), 4c6169b10f2e4152be4f90c81836aadb (127.13.75.131:38233)
Sep 23 15:05:21 dist-test-slave-cnb0 krb5kdc[28944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175919, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.129@KRBTEST.COM for kudu/127.13.75.131@KRBTEST.COM
Sep 23 15:05:21 dist-test-slave-cnb0 krb5kdc[28944](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175919, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.129@KRBTEST.COM for kudu/127.13.75.130@KRBTEST.COM
I20260923 15:05:21.495075 30525 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:21.484240 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.129:39893 (local address 127.13.75.131:38233)
0923 15:05:21.484370 (+ 130us) server_negotiation.cc:206] Beginning negotiation
0923 15:05:21.484379 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0923 15:05:21.484395 (+ 16us) server_negotiation.cc:407] Connection header received
0923 15:05:21.484818 (+ 423us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:05:21.484820 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:05:21.484859 (+ 39us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:05:21.485014 (+ 155us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:05:21.486007 (+ 993us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:05:21.486955 (+ 948us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:05:21.487823 (+ 868us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:05:21.488202 (+ 379us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:05:21.488322 (+ 120us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:05:21.490743 (+ 2421us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:05:21.490748 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:05:21.490748 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:05:21.490760 (+ 12us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:05:21.492786 (+ 2026us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:05:21.493473 (+ 687us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:05:21.493477 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:05:21.493477 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:05:21.493591 (+ 114us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:05:21.493961 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:05:21.493962 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:05:21.493963 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:05:21.494313 (+ 350us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:05:21.494399 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0923 15:05:21.494802 (+ 403us) server_negotiation.cc:299] Negotiation successful
0923 15:05:21.494958 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":102,"thread_start_us":57,"threads_started":1}
I20260923 15:05:21.495076 30523 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:21.484062 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.129:34347 (local address 127.13.75.130:44675)
0923 15:05:21.484280 (+ 218us) server_negotiation.cc:206] Beginning negotiation
0923 15:05:21.484293 (+ 13us) server_negotiation.cc:399] Waiting for connection header
0923 15:05:21.484319 (+ 26us) server_negotiation.cc:407] Connection header received
0923 15:05:21.484812 (+ 493us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:05:21.484814 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:05:21.484859 (+ 45us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:05:21.485008 (+ 149us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:05:21.485797 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:05:21.486741 (+ 944us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:05:21.487794 (+ 1053us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:05:21.488202 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:05:21.488336 (+ 134us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:05:21.491209 (+ 2873us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:05:21.491212 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:05:21.491212 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:05:21.491223 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:05:21.493242 (+ 2019us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:05:21.493786 (+ 544us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:05:21.493789 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:05:21.493789 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:05:21.493916 (+ 127us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:05:21.494118 (+ 202us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:05:21.494119 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:05:21.494119 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:05:21.494451 (+ 332us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:05:21.494522 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0923 15:05:21.494802 (+ 280us) server_negotiation.cc:299] Negotiation successful
0923 15:05:21.494958 (+ 156us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":177,"thread_start_us":68,"threads_started":1}
I20260923 15:05:21.495077 30524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:21.484111 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.131:38233 (local address 127.13.75.129:39893)
0923 15:05:21.484208 (+ 97us) negotiation.cc:107] Waiting for socket to connect
0923 15:05:21.484219 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 15:05:21.484270 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:05:21.485052 (+ 782us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:05:21.485055 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:05:21.485446 (+ 391us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:05:21.485894 (+ 448us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:05:21.485898 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:05:21.487032 (+ 1134us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:05:21.487033 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:05:21.487700 (+ 667us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:05:21.487702 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:05:21.488290 (+ 588us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:05:21.488291 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:05:21.488419 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:05:21.489055 (+ 636us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:05:21.489064 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:05:21.490629 (+ 1565us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:05:21.492873 (+ 2244us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:05:21.492880 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:05:21.492883 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:05:21.493237 (+ 354us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:05:21.493692 (+ 455us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:05:21.493693 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:05:21.493693 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:05:21.493876 (+ 183us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:05:21.494401 (+ 525us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:05:21.494408 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:05:21.494576 (+ 168us) client_negotiation.cc:769] Sending connection context
0923 15:05:21.494713 (+ 137us) client_negotiation.cc:240] Negotiation successful
0923 15:05:21.494922 (+ 209us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":84,"mutex_wait_us":26,"thread_start_us":47,"threads_started":1}
I20260923 15:05:21.495093 30522 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:21.483921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.130:44675 (local address 127.13.75.129:34347)
0923 15:05:21.484087 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0923 15:05:21.484102 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0923 15:05:21.484173 (+ 71us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:05:21.485053 (+ 880us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:05:21.485056 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:05:21.485443 (+ 387us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:05:21.485665 (+ 222us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:05:21.485671 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:05:21.486840 (+ 1169us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:05:21.486841 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:05:21.487701 (+ 860us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:05:21.487703 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:05:21.488305 (+ 602us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:05:21.488306 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:05:21.488420 (+ 114us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:05:21.489130 (+ 710us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:05:21.489134 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:05:21.491084 (+ 1950us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:05:21.493464 (+ 2380us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:05:21.493465 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:05:21.493466 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:05:21.493694 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:05:21.493995 (+ 301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:05:21.493995 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:05:21.493996 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:05:21.494052 (+ 56us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:05:21.494538 (+ 486us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:05:21.494539 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:05:21.494577 (+ 38us) client_negotiation.cc:769] Sending connection context
0923 15:05:21.494713 (+ 136us) client_negotiation.cc:240] Negotiation successful
0923 15:05:21.494975 (+ 262us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":147,"mutex_wait_us":116,"thread_start_us":76,"threads_started":1}
W20260923 15:05:21.496013 30136 leader_election.cc:341] T fa2002185f6a4877a2def06049cf198b P 125b34b86b954465bc1945b9733a2b0a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer fd45cc8453994d0fabe2a44f19a74598 (127.13.75.130:44675): Not found: Tablet not found: fa2002185f6a4877a2def06049cf198b
W20260923 15:05:21.496497 30136 leader_election.cc:341] T fa2002185f6a4877a2def06049cf198b P 125b34b86b954465bc1945b9733a2b0a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4c6169b10f2e4152be4f90c81836aadb (127.13.75.131:38233): Not found: Tablet not found: fa2002185f6a4877a2def06049cf198b
I20260923 15:05:21.496623 30136 leader_election.cc:302] T fa2002185f6a4877a2def06049cf198b P 125b34b86b954465bc1945b9733a2b0a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 125b34b86b954465bc1945b9733a2b0a; no voters: 4c6169b10f2e4152be4f90c81836aadb, fd45cc8453994d0fabe2a44f19a74598
I20260923 15:05:21.496770 30521 raft_consensus.cc:2726] T fa2002185f6a4877a2def06049cf198b P 125b34b86b954465bc1945b9733a2b0a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260923 15:05:21.751396 30524 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:21.751257 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:40941 (local address 127.13.75.129:33125)
0923 15:05:21.751315 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0923 15:05:21.751354 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.190:40941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260923 15:05:21.827368 30526 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:21.827161 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:40941 (local address 127.13.75.130:36945)
0923 15:05:21.827288 (+ 127us) negotiation.cc:107] Waiting for socket to connect
0923 15:05:21.827326 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.190:40941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":72,"threads_started":1}
I20260923 15:05:21.955533 30527 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:21.955276 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:40941 (local address 127.13.75.131:39211)
0923 15:05:21.955434 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0923 15:05:21.955486 (+ 52us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.190:40941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":74,"threads_started":1}
I20260923 15:05:22.752251 30528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:22.751989 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:40941 (local address 127.13.75.129:54429)
0923 15:05:22.752139 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0923 15:05:22.752185 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.190:40941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":129,"thread_start_us":64,"threads_started":1}
I20260923 15:05:22.828024 30529 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:22.827805 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:40941 (local address 127.13.75.130:43965)
0923 15:05:22.827937 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0923 15:05:22.827982 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.190:40941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":63,"threads_started":1}
I20260923 15:05:22.956321 30530 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:22.956065 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:40941 (local address 127.13.75.131:42165)
0923 15:05:22.956240 (+ 175us) negotiation.cc:107] Waiting for socket to connect
0923 15:05:22.956277 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.190:40941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":156,"thread_start_us":69,"threads_started":1}
I20260923 15:05:23.156077 30531 raft_consensus.cc:482] T fa2002185f6a4877a2def06049cf198b P 125b34b86b954465bc1945b9733a2b0a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 15:05:23.156215 30531 raft_consensus.cc:504] T fa2002185f6a4877a2def06049cf198b P 125b34b86b954465bc1945b9733a2b0a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd45cc8453994d0fabe2a44f19a74598" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 44675 } } peers { permanent_uuid: "4c6169b10f2e4152be4f90c81836aadb" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 38233 } } peers { permanent_uuid: "125b34b86b954465bc1945b9733a2b0a" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 39471 } }
I20260923 15:05:23.156342 30531 leader_election.cc:288] T fa2002185f6a4877a2def06049cf198b P 125b34b86b954465bc1945b9733a2b0a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fd45cc8453994d0fabe2a44f19a74598 (127.13.75.130:44675), 4c6169b10f2e4152be4f90c81836aadb (127.13.75.131:38233)
W20260923 15:05:23.156900 30136 leader_election.cc:341] T fa2002185f6a4877a2def06049cf198b P 125b34b86b954465bc1945b9733a2b0a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer fd45cc8453994d0fabe2a44f19a74598 (127.13.75.130:44675): Not found: Tablet not found: fa2002185f6a4877a2def06049cf198b
W20260923 15:05:23.156972 30136 leader_election.cc:341] T fa2002185f6a4877a2def06049cf198b P 125b34b86b954465bc1945b9733a2b0a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4c6169b10f2e4152be4f90c81836aadb (127.13.75.131:38233): Not found: Tablet not found: fa2002185f6a4877a2def06049cf198b
I20260923 15:05:23.157013 30136 leader_election.cc:302] T fa2002185f6a4877a2def06049cf198b P 125b34b86b954465bc1945b9733a2b0a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 125b34b86b954465bc1945b9733a2b0a; no voters: 4c6169b10f2e4152be4f90c81836aadb, fd45cc8453994d0fabe2a44f19a74598
I20260923 15:05:23.157116 30531 raft_consensus.cc:2726] T fa2002185f6a4877a2def06049cf198b P 125b34b86b954465bc1945b9733a2b0a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260923 15:05:23.752956 30532 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:23.752740 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:40941 (local address 127.13.75.129:53713)
0923 15:05:23.752871 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0923 15:05:23.752907 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.190:40941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":63,"threads_started":1}
I20260923 15:05:23.828715 30533 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:23.828513 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:40941 (local address 127.13.75.130:40863)
0923 15:05:23.828635 (+ 122us) negotiation.cc:107] Waiting for socket to connect
0923 15:05:23.828673 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.190:40941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":105,"thread_start_us":61,"threads_started":1}
I20260923 15:05:23.957163 30534 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:23.956840 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:40941 (local address 127.13.75.131:59069)
0923 15:05:23.957024 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0923 15:05:23.957072 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.190:40941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":147,"thread_start_us":64,"threads_started":1}
I20260923 15:05:24.753777 30535 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:24.753534 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:40941 (local address 127.13.75.129:40955)
0923 15:05:24.753678 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0923 15:05:24.753723 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.190:40941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":64,"threads_started":1}
I20260923 15:05:24.829469 30536 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:24.829265 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:40941 (local address 127.13.75.130:54587)
0923 15:05:24.829381 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0923 15:05:24.829418 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.190:40941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":91,"thread_start_us":63,"threads_started":1}
I20260923 15:05:24.957927 30537 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:24.957702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:40941 (local address 127.13.75.131:59687)
0923 15:05:24.957841 (+ 139us) negotiation.cc:107] Waiting for socket to connect
0923 15:05:24.957885 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.190:40941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":121,"thread_start_us":70,"threads_started":1}
I20260923 15:05:25.754647 30538 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:25.754408 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:40941 (local address 127.13.75.129:59219)
0923 15:05:25.754553 (+ 145us) negotiation.cc:107] Waiting for socket to connect
0923 15:05:25.754590 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.190:40941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":120,"thread_start_us":85,"threads_started":1}
I20260923 15:05:25.830220 30539 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:25.829979 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:40941 (local address 127.13.75.130:33037)
0923 15:05:25.830129 (+ 150us) negotiation.cc:107] Waiting for socket to connect
0923 15:05:25.830172 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.190:40941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":59,"threads_started":1}
I20260923 15:05:25.958706 30540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:25.958488 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:40941 (local address 127.13.75.131:54851)
0923 15:05:25.958617 (+ 129us) negotiation.cc:107] Waiting for socket to connect
0923 15:05:25.958656 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.190:40941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":59,"threads_started":1}
I20260923 15:05:26.414273 30541 raft_consensus.cc:482] T fa2002185f6a4877a2def06049cf198b P 125b34b86b954465bc1945b9733a2b0a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 15:05:26.414390 30541 raft_consensus.cc:504] T fa2002185f6a4877a2def06049cf198b P 125b34b86b954465bc1945b9733a2b0a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd45cc8453994d0fabe2a44f19a74598" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 44675 } } peers { permanent_uuid: "4c6169b10f2e4152be4f90c81836aadb" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 38233 } } peers { permanent_uuid: "125b34b86b954465bc1945b9733a2b0a" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 39471 } }
I20260923 15:05:26.414580 30541 leader_election.cc:288] T fa2002185f6a4877a2def06049cf198b P 125b34b86b954465bc1945b9733a2b0a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fd45cc8453994d0fabe2a44f19a74598 (127.13.75.130:44675), 4c6169b10f2e4152be4f90c81836aadb (127.13.75.131:38233)
W20260923 15:05:26.414948 30136 leader_election.cc:341] T fa2002185f6a4877a2def06049cf198b P 125b34b86b954465bc1945b9733a2b0a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer fd45cc8453994d0fabe2a44f19a74598 (127.13.75.130:44675): Not found: Tablet not found: fa2002185f6a4877a2def06049cf198b
W20260923 15:05:26.415035 30136 leader_election.cc:341] T fa2002185f6a4877a2def06049cf198b P 125b34b86b954465bc1945b9733a2b0a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4c6169b10f2e4152be4f90c81836aadb (127.13.75.131:38233): Not found: Tablet not found: fa2002185f6a4877a2def06049cf198b
I20260923 15:05:26.415052 30136 leader_election.cc:302] T fa2002185f6a4877a2def06049cf198b P 125b34b86b954465bc1945b9733a2b0a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 125b34b86b954465bc1945b9733a2b0a; no voters: 4c6169b10f2e4152be4f90c81836aadb, fd45cc8453994d0fabe2a44f19a74598
I20260923 15:05:26.415120 30541 raft_consensus.cc:2726] T fa2002185f6a4877a2def06049cf198b P 125b34b86b954465bc1945b9733a2b0a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260923 15:05:26.755383 30542 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:26.755122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:40941 (local address 127.13.75.129:58457)
0923 15:05:26.755291 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0923 15:05:26.755332 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.190:40941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":61,"threads_started":1}
I20260923 15:05:26.831027 30543 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:26.830734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:40941 (local address 127.13.75.130:35471)
0923 15:05:26.830874 (+ 140us) negotiation.cc:107] Waiting for socket to connect
0923 15:05:26.830978 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.190:40941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":122,"thread_start_us":66,"threads_started":1}
I20260923 15:05:26.959475 30544 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:26.959220 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:40941 (local address 127.13.75.131:39651)
0923 15:05:26.959389 (+ 169us) negotiation.cc:107] Waiting for socket to connect
0923 15:05:26.959430 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.190:40941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":149,"thread_start_us":101,"threads_started":1}
I20260923 15:05:27.756115 30545 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:27.755895 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:40941 (local address 127.13.75.129:40659)
0923 15:05:27.756023 (+ 128us) negotiation.cc:107] Waiting for socket to connect
0923 15:05:27.756065 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.190:40941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":63,"threads_started":1}
I20260923 15:05:27.831910 30546 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:27.831561 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:40941 (local address 127.13.75.130:56301)
0923 15:05:27.831732 (+ 171us) negotiation.cc:107] Waiting for socket to connect
0923 15:05:27.831774 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.190:40941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":80,"threads_started":1}
I20260923 15:05:27.960139 30547 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:27.959925 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:40941 (local address 127.13.75.131:40131)
0923 15:05:27.960056 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0923 15:05:27.960092 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.190:40941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":109,"thread_start_us":59,"threads_started":1}
I20260923 15:05:28.756871 30548 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:28.756628 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:40941 (local address 127.13.75.129:46493)
0923 15:05:28.756781 (+ 153us) negotiation.cc:107] Waiting for socket to connect
0923 15:05:28.756823 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.190:40941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":65,"threads_started":1}
I20260923 15:05:28.832602 30549 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:28.832360 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:40941 (local address 127.13.75.130:35671)
0923 15:05:28.832512 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0923 15:05:28.832556 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.190:40941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":133,"thread_start_us":81,"threads_started":1}
I20260923 15:05:28.960914 30550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:28.960665 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:40941 (local address 127.13.75.131:39855)
0923 15:05:28.960821 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0923 15:05:28.960867 (+ 46us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.190:40941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":127,"thread_start_us":63,"threads_started":1}
I20260923 15:05:29.059199 30551 raft_consensus.cc:482] T fa2002185f6a4877a2def06049cf198b P 125b34b86b954465bc1945b9733a2b0a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 15:05:29.059294 30551 raft_consensus.cc:504] T fa2002185f6a4877a2def06049cf198b P 125b34b86b954465bc1945b9733a2b0a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd45cc8453994d0fabe2a44f19a74598" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 44675 } } peers { permanent_uuid: "4c6169b10f2e4152be4f90c81836aadb" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 38233 } } peers { permanent_uuid: "125b34b86b954465bc1945b9733a2b0a" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 39471 } }
I20260923 15:05:29.059417 30551 leader_election.cc:288] T fa2002185f6a4877a2def06049cf198b P 125b34b86b954465bc1945b9733a2b0a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fd45cc8453994d0fabe2a44f19a74598 (127.13.75.130:44675), 4c6169b10f2e4152be4f90c81836aadb (127.13.75.131:38233)
W20260923 15:05:29.059868 30136 leader_election.cc:341] T fa2002185f6a4877a2def06049cf198b P 125b34b86b954465bc1945b9733a2b0a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer fd45cc8453994d0fabe2a44f19a74598 (127.13.75.130:44675): Not found: Tablet not found: fa2002185f6a4877a2def06049cf198b
W20260923 15:05:29.059975 30136 leader_election.cc:341] T fa2002185f6a4877a2def06049cf198b P 125b34b86b954465bc1945b9733a2b0a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4c6169b10f2e4152be4f90c81836aadb (127.13.75.131:38233): Not found: Tablet not found: fa2002185f6a4877a2def06049cf198b
I20260923 15:05:29.060012 30136 leader_election.cc:302] T fa2002185f6a4877a2def06049cf198b P 125b34b86b954465bc1945b9733a2b0a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 125b34b86b954465bc1945b9733a2b0a; no voters: 4c6169b10f2e4152be4f90c81836aadb, fd45cc8453994d0fabe2a44f19a74598
I20260923 15:05:29.060103 30551 raft_consensus.cc:2726] T fa2002185f6a4877a2def06049cf198b P 125b34b86b954465bc1945b9733a2b0a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260923 15:05:29.757725 30552 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:29.757508 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:40941 (local address 127.13.75.129:35803)
0923 15:05:29.757633 (+ 125us) negotiation.cc:107] Waiting for socket to connect
0923 15:05:29.757675 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.190:40941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":108,"thread_start_us":65,"threads_started":1}
I20260923 15:05:29.833351 30553 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:29.833092 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:40941 (local address 127.13.75.130:59005)
0923 15:05:29.833244 (+ 152us) negotiation.cc:107] Waiting for socket to connect
0923 15:05:29.833300 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.190:40941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":130,"thread_start_us":61,"threads_started":1}
I20260923 15:05:29.961817 30554 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:29.961458 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:40941 (local address 127.13.75.131:58665)
0923 15:05:29.961694 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0923 15:05:29.961749 (+ 55us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.190:40941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":210,"thread_start_us":97,"threads_started":1}
I20260923 15:05:30.758457 30555 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:30.758219 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:40941 (local address 127.13.75.129:36043)
0923 15:05:30.758365 (+ 146us) negotiation.cc:107] Waiting for socket to connect
0923 15:05:30.758406 (+ 41us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.190:40941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":128,"thread_start_us":65,"threads_started":1}
I20260923 15:05:30.834151 30556 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:30.833929 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:40941 (local address 127.13.75.130:35745)
0923 15:05:30.834065 (+ 136us) negotiation.cc:107] Waiting for socket to connect
0923 15:05:30.834104 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.190:40941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":78,"threads_started":1}
I20260923 15:05:30.962636 30557 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:30.962348 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:40941 (local address 127.13.75.131:40335)
0923 15:05:30.962516 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0923 15:05:30.962586 (+ 70us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.190:40941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":142,"thread_start_us":76,"threads_started":1}
I20260923 15:05:31.759320 30558 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:31.759057 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:40941 (local address 127.13.75.129:38869)
0923 15:05:31.759222 (+ 165us) negotiation.cc:107] Waiting for socket to connect
0923 15:05:31.759269 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.190:40941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":143,"thread_start_us":66,"threads_started":1}
I20260923 15:05:31.835018 30559 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:31.834713 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:40941 (local address 127.13.75.130:53025)
0923 15:05:31.834869 (+ 156us) negotiation.cc:107] Waiting for socket to connect
0923 15:05:31.834973 (+ 104us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.190:40941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":137,"thread_start_us":71,"threads_started":1}
I20260923 15:05:31.963370 30560 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:31.963105 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:40941 (local address 127.13.75.131:59607)
0923 15:05:31.963284 (+ 179us) negotiation.cc:107] Waiting for socket to connect
0923 15:05:31.963324 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.190:40941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":162,"thread_start_us":116,"threads_started":1}
I20260923 15:05:32.299904 30561 raft_consensus.cc:482] T fa2002185f6a4877a2def06049cf198b P 125b34b86b954465bc1945b9733a2b0a [term 1 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 15:05:32.299999 30561 raft_consensus.cc:504] T fa2002185f6a4877a2def06049cf198b P 125b34b86b954465bc1945b9733a2b0a [term 1 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "fd45cc8453994d0fabe2a44f19a74598" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 44675 } } peers { permanent_uuid: "4c6169b10f2e4152be4f90c81836aadb" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 38233 } } peers { permanent_uuid: "125b34b86b954465bc1945b9733a2b0a" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 39471 } }
I20260923 15:05:32.300132 30561 leader_election.cc:288] T fa2002185f6a4877a2def06049cf198b P 125b34b86b954465bc1945b9733a2b0a [CANDIDATE]: Term 2 pre-election: Requested pre-vote from peers fd45cc8453994d0fabe2a44f19a74598 (127.13.75.130:44675), 4c6169b10f2e4152be4f90c81836aadb (127.13.75.131:38233)
W20260923 15:05:32.300668 30136 leader_election.cc:341] T fa2002185f6a4877a2def06049cf198b P 125b34b86b954465bc1945b9733a2b0a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer fd45cc8453994d0fabe2a44f19a74598 (127.13.75.130:44675): Not found: Tablet not found: fa2002185f6a4877a2def06049cf198b
W20260923 15:05:32.300743 30136 leader_election.cc:341] T fa2002185f6a4877a2def06049cf198b P 125b34b86b954465bc1945b9733a2b0a [CANDIDATE]: Term 2 pre-election: Tablet error from VoteRequest() call to peer 4c6169b10f2e4152be4f90c81836aadb (127.13.75.131:38233): Not found: Tablet not found: fa2002185f6a4877a2def06049cf198b
I20260923 15:05:32.300792 30136 leader_election.cc:302] T fa2002185f6a4877a2def06049cf198b P 125b34b86b954465bc1945b9733a2b0a [CANDIDATE]: Term 2 pre-election: Election decided. Result: candidate lost. Election summary: received 3 responses out of 3 voters: 1 yes votes; 2 no votes. yes voters: 125b34b86b954465bc1945b9733a2b0a; no voters: 4c6169b10f2e4152be4f90c81836aadb, fd45cc8453994d0fabe2a44f19a74598
I20260923 15:05:32.300889 30561 raft_consensus.cc:2726] T fa2002185f6a4877a2def06049cf198b P 125b34b86b954465bc1945b9733a2b0a [term 1 FOLLOWER]: Leader pre-election lost for term 2. Reason: could not achieve majority
I20260923 15:05:32.760174 30562 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:32.759953 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:40941 (local address 127.13.75.129:35247)
0923 15:05:32.760086 (+ 133us) negotiation.cc:107] Waiting for socket to connect
0923 15:05:32.760124 (+ 38us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.190:40941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":114,"thread_start_us":64,"threads_started":1}
I20260923 15:05:32.835825 30563 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:32.835567 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:40941 (local address 127.13.75.130:41177)
0923 15:05:32.835733 (+ 166us) negotiation.cc:107] Waiting for socket to connect
0923 15:05:32.835775 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.190:40941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":146,"thread_start_us":76,"threads_started":1}
I20260923 15:05:32.964149 30564 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:32.963819 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:40941 (local address 127.13.75.131:43461)
0923 15:05:32.963989 (+ 170us) negotiation.cc:107] Waiting for socket to connect
0923 15:05:32.964058 (+ 69us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.190:40941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":61,"threads_started":1}
I20260923 15:05:33.761031 30565 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:33.760775 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:40941 (local address 127.13.75.129:47091)
0923 15:05:33.760919 (+ 144us) negotiation.cc:107] Waiting for socket to connect
0923 15:05:33.760966 (+ 47us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.190:40941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":123,"thread_start_us":65,"threads_started":1}
I20260923 15:05:33.836625 30566 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:33.836403 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:40941 (local address 127.13.75.130:52121)
0923 15:05:33.836534 (+ 131us) negotiation.cc:107] Waiting for socket to connect
0923 15:05:33.836579 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.190:40941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":112,"thread_start_us":63,"threads_started":1}
I20260923 15:05:33.964920 30567 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:33.964688 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:40941 (local address 127.13.75.131:52177)
0923 15:05:33.964830 (+ 142us) negotiation.cc:107] Waiting for socket to connect
0923 15:05:33.964874 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.190:40941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":124,"thread_start_us":72,"threads_started":1}
I20260923 15:05:34.761945 30568 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:34.761697 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:40941 (local address 127.13.75.129:44289)
0923 15:05:34.761857 (+ 160us) negotiation.cc:107] Waiting for socket to connect
0923 15:05:34.761894 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.190:40941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":70,"threads_started":1}
I20260923 15:05:34.837432 30569 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:34.837196 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:40941 (local address 127.13.75.130:60371)
0923 15:05:34.837350 (+ 154us) negotiation.cc:107] Waiting for socket to connect
0923 15:05:34.837385 (+ 35us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.190:40941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":139,"thread_start_us":61,"threads_started":1}
I20260923 15:05:34.960796 13614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKC4soy/build/release/bin/kudu with pid 30083
I20260923 15:05:34.965840 30570 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:34.965574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:40941 (local address 127.13.75.131:44233)
0923 15:05:34.965737 (+ 163us) negotiation.cc:107] Waiting for socket to connect
0923 15:05:34.965785 (+ 48us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.190:40941: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":144,"thread_start_us":91,"threads_started":1}
I20260923 15:05:34.966017 13614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKC4soy/build/release/bin/kudu with pid 30252
I20260923 15:05:34.970655 13614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKC4soy/build/release/bin/kudu with pid 30385
2026-09-23T15:05:34Z chronyd exiting
[ OK ] SecurityITest.TestEnableAndDisableEncryptedTSK (58300 ms)
[----------] 30 tests from SecurityITest (243486 ms total)
[----------] 6 tests from EncryptionPolicyTest
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/0
Loading random data
Initializing database '/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175691491353-13614-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 15:05:35 dist-test-slave-cnb0 krb5kdc[30573](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 15:05:35 dist-test-slave-cnb0 krb5kdc[30573](info): set up 2 sockets
Sep 23 15:05:35 dist-test-slave-cnb0 krb5kdc[30573](info): commencing operation
krb5kdc: starting...
W20260923 15:05:36.048568 13614 mini_kdc.cc:121] Time spent starting KDC: real 1.056s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175691491353-13614-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175691491353-13614-0/krb5kdc/test-user.keytab.
Sep 23 15:05:36 dist-test-slave-cnb0 krb5kdc[30573](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175936, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T15:05:36Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T15:05:36Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.75.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.75.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.75.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.190.keytab.
Entry for principal kudu/127.13.75.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.190.keytab.
I20260923 15:05:36.178079 13614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKC4soy/build/release/bin/kudu
/tmp/dist-test-taskKC4soy/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175691491353-13614-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175691491353-13614-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175691491353-13614-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175691491353-13614-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.75.190:41925
--webserver_interface=127.13.75.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.190.keytab
--principal=kudu/127.13.75.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.75.148:39491
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.75.190:41925
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175691491353-13614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175691491353-13614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175691491353-13614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:05:36.260893 30586 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:05:36.261163 30586 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:05:36.261204 30586 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:05:36.262395 30586 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 15:05:36.262473 30586 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:05:36.262498 30586 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:05:36.262527 30586 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 15:05:36.262558 30586 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 15:05:36.264163 30586 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.75.148:39491
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175691491353-13614-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175691491353-13614-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.75.190:41925
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.190.keytab
--principal=kudu/127.13.75.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.75.190:41925
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175691491353-13614-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.75.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30586
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175691491353-13614-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:05:36.264474 30586 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:05:36.264717 30586 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260923 15:05:36.270608 30586 server_base.cc:1034] running on GCE node
W20260923 15:05:36.270705 30591 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:05:36.270654 30594 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:05:36.270759 30592 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:05:36.271291 30586 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:05:36.271624 30586 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:05:36.272814 30586 hybrid_clock.cc:648] HybridClock initialized: now 1790175936272795 us; error 44 us; skew 500 ppm
Sep 23 15:05:36 dist-test-slave-cnb0 krb5kdc[30573](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175936, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:05:36.275523 30586 init.cc:377] Logged in from keytab as kudu/127.13.75.190@KRBTEST.COM (short username kudu)
I20260923 15:05:36.275966 30586 webserver.cc:460] Webserver started at http://127.13.75.190:38697/ using document root <none> and password file <none>
I20260923 15:05:36.276216 30586 fs_manager.cc:359] Metadata directory not provided
I20260923 15:05:36.276264 30586 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:05:36.276397 30586 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:05:36.277537 30586 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175691491353-13614-0/minicluster-data/master-0/data/instance:
uuid: "9fd42eea14af4c2b9e47ec30f042b9af"
format_stamp: "Formatted at 2026-09-23 15:05:36 on dist-test-slave-cnb0"
server_key: "1979b5de76c03e068b56a26486452da7"
server_key_iv: "b96663f05e67e7b4a8789a8b3877c49e"
server_key_version: "encryptionkey@0"
I20260923 15:05:36.277925 30586 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175691491353-13614-0/minicluster-data/master-0/wal/instance:
uuid: "9fd42eea14af4c2b9e47ec30f042b9af"
format_stamp: "Formatted at 2026-09-23 15:05:36 on dist-test-slave-cnb0"
server_key: "1979b5de76c03e068b56a26486452da7"
server_key_iv: "b96663f05e67e7b4a8789a8b3877c49e"
server_key_version: "encryptionkey@0"
I20260923 15:05:36.279531 30586 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260923 15:05:36.280570 30601 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:05:36.280886 30586 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260923 15:05:36.280982 30586 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175691491353-13614-0/minicluster-data/master-0/data,/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175691491353-13614-0/minicluster-data/master-0/wal
uuid: "9fd42eea14af4c2b9e47ec30f042b9af"
format_stamp: "Formatted at 2026-09-23 15:05:36 on dist-test-slave-cnb0"
server_key: "1979b5de76c03e068b56a26486452da7"
server_key_iv: "b96663f05e67e7b4a8789a8b3877c49e"
server_key_version: "encryptionkey@0"
I20260923 15:05:36.281091 30586 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175691491353-13614-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175691491353-13614-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175691491353-13614-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:05:36.296864 30586 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:05:36.299361 30586 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:05:36.299518 30586 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:05:36.303555 30586 rpc_server.cc:307] RPC server started. Bound to: 127.13.75.190:41925
I20260923 15:05:36.303627 30653 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.75.190:41925 every 8 connection(s)
I20260923 15:05:36.304025 30586 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175691491353-13614-0/minicluster-data/master-0/data/info.pb
I20260923 15:05:36.304939 30654 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:05:36.307327 30654 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9fd42eea14af4c2b9e47ec30f042b9af: Bootstrap starting.
I20260923 15:05:36.307952 30654 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9fd42eea14af4c2b9e47ec30f042b9af: Neither blocks nor log segments found. Creating new log.
I20260923 15:05:36.308279 30654 log.cc:824] T 00000000000000000000000000000000 P 9fd42eea14af4c2b9e47ec30f042b9af: Log is configured to *not* fsync() on all Append() calls
I20260923 15:05:36.309079 30654 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9fd42eea14af4c2b9e47ec30f042b9af: No bootstrap required, opened a new log
I20260923 15:05:36.310792 30654 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9fd42eea14af4c2b9e47ec30f042b9af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fd42eea14af4c2b9e47ec30f042b9af" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 41925 } }
I20260923 15:05:36.310994 30654 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9fd42eea14af4c2b9e47ec30f042b9af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:05:36.311019 30654 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9fd42eea14af4c2b9e47ec30f042b9af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9fd42eea14af4c2b9e47ec30f042b9af, State: Initialized, Role: FOLLOWER
I20260923 15:05:36.311203 30654 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9fd42eea14af4c2b9e47ec30f042b9af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fd42eea14af4c2b9e47ec30f042b9af" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 41925 } }
I20260923 15:05:36.311273 30654 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9fd42eea14af4c2b9e47ec30f042b9af [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 15:05:36.311318 30654 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9fd42eea14af4c2b9e47ec30f042b9af [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 15:05:36.311367 30654 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9fd42eea14af4c2b9e47ec30f042b9af [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:05:36.311944 30654 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9fd42eea14af4c2b9e47ec30f042b9af [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fd42eea14af4c2b9e47ec30f042b9af" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 41925 } }
I20260923 15:05:36.312029 30654 leader_election.cc:302] T 00000000000000000000000000000000 P 9fd42eea14af4c2b9e47ec30f042b9af [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9fd42eea14af4c2b9e47ec30f042b9af; no voters:
I20260923 15:05:36.312223 30654 leader_election.cc:288] T 00000000000000000000000000000000 P 9fd42eea14af4c2b9e47ec30f042b9af [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 15:05:36.312275 30657 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9fd42eea14af4c2b9e47ec30f042b9af [term 1 FOLLOWER]: Leader election won for term 1
I20260923 15:05:36.312474 30657 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9fd42eea14af4c2b9e47ec30f042b9af [term 1 LEADER]: Becoming Leader. State: Replica: 9fd42eea14af4c2b9e47ec30f042b9af, State: Running, Role: LEADER
I20260923 15:05:36.312561 30654 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9fd42eea14af4c2b9e47ec30f042b9af [sys.catalog]: configured and running, proceeding with master startup.
I20260923 15:05:36.312641 30657 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9fd42eea14af4c2b9e47ec30f042b9af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fd42eea14af4c2b9e47ec30f042b9af" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 41925 } }
I20260923 15:05:36.313043 30659 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9fd42eea14af4c2b9e47ec30f042b9af [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9fd42eea14af4c2b9e47ec30f042b9af. Latest consensus state: current_term: 1 leader_uuid: "9fd42eea14af4c2b9e47ec30f042b9af" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fd42eea14af4c2b9e47ec30f042b9af" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 41925 } } }
I20260923 15:05:36.313126 30659 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9fd42eea14af4c2b9e47ec30f042b9af [sys.catalog]: This master's current role is: LEADER
I20260923 15:05:36.313179 30658 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9fd42eea14af4c2b9e47ec30f042b9af [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9fd42eea14af4c2b9e47ec30f042b9af" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9fd42eea14af4c2b9e47ec30f042b9af" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 41925 } } }
I20260923 15:05:36.313246 30658 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9fd42eea14af4c2b9e47ec30f042b9af [sys.catalog]: This master's current role is: LEADER
I20260923 15:05:36.313428 30666 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 15:05:36.313786 13614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKC4soy/build/release/bin/kudu as pid 30586
I20260923 15:05:36.313889 13614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175691491353-13614-0/minicluster-data/master-0/wal/instance
I20260923 15:05:36.314043 30666 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 15:05:36.314126 13614 external_mini_cluster.cc:1428] Setting key 33539ff45cea142ca17c884eac6f078d
I20260923 15:05:36.315802 30666 catalog_manager.cc:1347] Generated new cluster ID: a626fceb4cc240c191322a511e370657
I20260923 15:05:36.315856 30666 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 23 15:05:36 dist-test-slave-cnb0 krb5kdc[30573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175936, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.75.190@KRBTEST.COM
I20260923 15:05:36.326095 30676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:36.314558 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34638 (local address 127.13.75.190:41925)
0923 15:05:36.314747 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0923 15:05:36.314759 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0923 15:05:36.314792 (+ 33us) server_negotiation.cc:407] Connection header received
0923 15:05:36.314934 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:05:36.314941 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:05:36.315007 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:05:36.315143 (+ 136us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:05:36.316005 (+ 862us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:05:36.317383 (+ 1378us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:05:36.318513 (+ 1130us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:05:36.319159 (+ 646us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:05:36.319259 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:05:36.322157 (+ 2898us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:05:36.322162 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:05:36.322163 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:05:36.322177 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:05:36.324314 (+ 2137us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:05:36.324729 (+ 415us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:05:36.324739 (+ 10us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:05:36.324739 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:05:36.324877 (+ 138us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:05:36.325084 (+ 207us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:05:36.325085 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:05:36.325085 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:05:36.325433 (+ 348us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:05:36.325598 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0923 15:05:36.325678 (+ 80us) server_negotiation.cc:299] Negotiation successful
0923 15:05:36.325835 (+ 157us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":142,"thread_start_us":81,"threads_started":1}
I20260923 15:05:36.334649 30666 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 15:05:36.335296 30666 catalog_manager.cc:1504] Loading token signing keys...
I20260923 15:05:36.339947 30666 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9fd42eea14af4c2b9e47ec30f042b9af: Generated new TSK 0
I20260923 15:05:36.340174 30666 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.75.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.75.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.75.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.129.keytab.
Entry for principal kudu/127.13.75.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.129.keytab.
I20260923 15:05:36.378651 13614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKC4soy/build/release/bin/kudu
/tmp/dist-test-taskKC4soy/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175691491353-13614-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175691491353-13614-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175691491353-13614-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175691491353-13614-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.75.129:0
--local_ip_for_outbound_sockets=127.13.75.129
--webserver_interface=127.13.75.129
--webserver_port=0
--tserver_master_addrs=127.13.75.190:41925
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.129.keytab
--principal=kudu/127.13.75.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.75.148:39491
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175691491353-13614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175691491353-13614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175691491353-13614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:05:36.464221 30680 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:05:36.464448 30680 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:05:36.464475 30680 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:05:36.465694 30680 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:05:36.465744 30680 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:05:36.465777 30680 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.75.129
I20260923 15:05:36.467409 30680 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.75.148:39491
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175691491353-13614-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175691491353-13614-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.129.keytab
--principal=kudu/127.13.75.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.75.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175691491353-13614-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.75.129
--webserver_port=0
--tserver_master_addrs=127.13.75.190:41925
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30680
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.75.129
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175691491353-13614-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:05:36.467682 30680 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:05:36.467949 30680 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:05:36.473321 30686 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:05:36.473387 30688 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:05:36.473393 30685 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:05:36.473631 30680 server_base.cc:1034] running on GCE node
I20260923 15:05:36.473887 30680 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:05:36.474155 30680 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:05:36.475308 30680 hybrid_clock.cc:648] HybridClock initialized: now 1790175936475284 us; error 36 us; skew 500 ppm
Sep 23 15:05:36 dist-test-slave-cnb0 krb5kdc[30573](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175936, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:05:36.479998 30680 init.cc:377] Logged in from keytab as kudu/127.13.75.129@KRBTEST.COM (short username kudu)
I20260923 15:05:36.480701 30680 webserver.cc:460] Webserver started at http://127.13.75.129:46847/ using document root <none> and password file <none>
I20260923 15:05:36.480999 30680 fs_manager.cc:359] Metadata directory not provided
I20260923 15:05:36.481092 30680 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:05:36.481266 30680 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:05:36.482630 30680 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175691491353-13614-0/minicluster-data/ts-0/data/instance:
uuid: "2b7a2526d96644fba233866d7768451d"
format_stamp: "Formatted at 2026-09-23 15:05:36 on dist-test-slave-cnb0"
server_key: "51bbbc180531e74627783d66b99c3b70"
server_key_iv: "f8742ca0a7d0dbc7a44eabd7fa4d748c"
server_key_version: "encryptionkey@0"
I20260923 15:05:36.483146 30680 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175691491353-13614-0/minicluster-data/ts-0/wal/instance:
uuid: "2b7a2526d96644fba233866d7768451d"
format_stamp: "Formatted at 2026-09-23 15:05:36 on dist-test-slave-cnb0"
server_key: "51bbbc180531e74627783d66b99c3b70"
server_key_iv: "f8742ca0a7d0dbc7a44eabd7fa4d748c"
server_key_version: "encryptionkey@0"
I20260923 15:05:36.484891 30680 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.002s
I20260923 15:05:36.485905 30695 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:05:36.486159 30680 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260923 15:05:36.486325 30680 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175691491353-13614-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175691491353-13614-0/minicluster-data/ts-0/wal
uuid: "2b7a2526d96644fba233866d7768451d"
format_stamp: "Formatted at 2026-09-23 15:05:36 on dist-test-slave-cnb0"
server_key: "51bbbc180531e74627783d66b99c3b70"
server_key_iv: "f8742ca0a7d0dbc7a44eabd7fa4d748c"
server_key_version: "encryptionkey@0"
I20260923 15:05:36.486413 30680 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175691491353-13614-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175691491353-13614-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175691491353-13614-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:05:36.497010 30680 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:05:36.499428 30680 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:05:36.499641 30680 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:05:36.499927 30680 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 15:05:36.500299 30680 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 15:05:36.500360 30680 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 15:05:36.500408 30680 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 15:05:36.500440 30680 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 15:05:36.507256 30680 rpc_server.cc:307] RPC server started. Bound to: 127.13.75.129:43387
I20260923 15:05:36.507331 30808 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.75.129:43387 every 8 connection(s)
I20260923 15:05:36.507706 30680 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175691491353-13614-0/minicluster-data/ts-0/data/info.pb
Sep 23 15:05:36 dist-test-slave-cnb0 krb5kdc[30573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175936, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.129@KRBTEST.COM for kudu/127.13.75.190@KRBTEST.COM
I20260923 15:05:36.514508 13614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKC4soy/build/release/bin/kudu as pid 30680
I20260923 15:05:36.514626 13614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175691491353-13614-0/minicluster-data/ts-0/wal/instance
I20260923 15:05:36.514865 13614 external_mini_cluster.cc:1428] Setting key 7b9196322f1bcd6c0d52174c93b6115a
I20260923 15:05:36.518051 30676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:36.508538 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.129:41407 (local address 127.13.75.190:41925)
0923 15:05:36.508618 (+ 80us) server_negotiation.cc:206] Beginning negotiation
0923 15:05:36.508621 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:05:36.508833 (+ 212us) server_negotiation.cc:407] Connection header received
0923 15:05:36.509380 (+ 547us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:05:36.509382 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:05:36.509392 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:05:36.509453 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:05:36.510418 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:05:36.511022 (+ 604us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:05:36.512088 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:05:36.512297 (+ 209us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:05:36.512354 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:05:36.514820 (+ 2466us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:05:36.514821 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:05:36.514822 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:05:36.514828 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:05:36.516611 (+ 1783us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:05:36.517048 (+ 437us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:05:36.517049 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:05:36.517050 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:05:36.517108 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:05:36.517383 (+ 275us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:05:36.517384 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:05:36.517384 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:05:36.517504 (+ 120us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:05:36.517578 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0923 15:05:36.517861 (+ 283us) server_negotiation.cc:299] Negotiation successful
0923 15:05:36.517958 (+ 97us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260923 15:05:36.518256 30811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:36.508526 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:41925 (local address 127.13.75.129:41407)
0923 15:05:36.508716 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0923 15:05:36.508762 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0923 15:05:36.509266 (+ 504us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:05:36.509514 (+ 248us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:05:36.509517 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:05:36.510093 (+ 576us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:05:36.510332 (+ 239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:05:36.510337 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:05:36.511107 (+ 770us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:05:36.511108 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:05:36.512012 (+ 904us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:05:36.512016 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:05:36.512375 (+ 359us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:05:36.512375 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:05:36.512478 (+ 103us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:05:36.513077 (+ 599us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:05:36.513088 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:05:36.514705 (+ 1617us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:05:36.516705 (+ 2000us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:05:36.516713 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:05:36.516715 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:05:36.516963 (+ 248us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:05:36.517184 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:05:36.517185 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:05:36.517185 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:05:36.517308 (+ 123us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:05:36.517568 (+ 260us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:05:36.517575 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:05:36.517723 (+ 148us) client_negotiation.cc:769] Sending connection context
0923 15:05:36.517839 (+ 116us) client_negotiation.cc:240] Negotiation successful
0923 15:05:36.518006 (+ 167us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":108,"threads_started":1}
I20260923 15:05:36.518760 30809 heartbeater.cc:347] Connected to a master server at 127.13.75.190:41925
I20260923 15:05:36.518862 30809 heartbeater.cc:464] Registering TS with master...
I20260923 15:05:36.519124 30809 heartbeater.cc:511] Master 127.13.75.190:41925 requested a full tablet report, sending...
I20260923 15:05:36.519820 30618 ts_manager.cc:194] Registered new tserver with Master: 2b7a2526d96644fba233866d7768451d (127.13.75.129:43387)
I20260923 15:05:36.520478 30618 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.75.129@KRBTEST.COM'} at 127.13.75.129:41407
WARNING: no policy specified for kudu/127.13.75.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.75.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.75.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.130.keytab.
Entry for principal kudu/127.13.75.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.130.keytab.
I20260923 15:05:36.543684 13614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKC4soy/build/release/bin/kudu
/tmp/dist-test-taskKC4soy/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175691491353-13614-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175691491353-13614-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175691491353-13614-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175691491353-13614-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.75.130:0
--local_ip_for_outbound_sockets=127.13.75.130
--webserver_interface=127.13.75.130
--webserver_port=0
--tserver_master_addrs=127.13.75.190:41925
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.130.keytab
--principal=kudu/127.13.75.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.75.148:39491
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175691491353-13614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175691491353-13614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175691491353-13614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:05:36.647063 30814 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:05:36.647374 30814 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:05:36.647418 30814 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:05:36.649217 30814 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:05:36.649286 30814 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:05:36.649343 30814 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.75.130
I20260923 15:05:36.651110 30814 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.75.148:39491
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175691491353-13614-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175691491353-13614-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.130.keytab
--principal=kudu/127.13.75.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.75.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175691491353-13614-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.75.130
--webserver_port=0
--tserver_master_addrs=127.13.75.190:41925
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30814
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.75.130
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175691491353-13614-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:05:36.651427 30814 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:05:36.651682 30814 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:05:36.657177 30822 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:05:36.657178 30819 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:05:36.657207 30820 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:05:36.657380 30814 server_base.cc:1034] running on GCE node
I20260923 15:05:36.657660 30814 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:05:36.658015 30814 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:05:36.659281 30814 hybrid_clock.cc:648] HybridClock initialized: now 1790175936659248 us; error 53 us; skew 500 ppm
Sep 23 15:05:36 dist-test-slave-cnb0 krb5kdc[30573](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175936, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:05:36.662302 30814 init.cc:377] Logged in from keytab as kudu/127.13.75.130@KRBTEST.COM (short username kudu)
I20260923 15:05:36.662835 30814 webserver.cc:460] Webserver started at http://127.13.75.130:37203/ using document root <none> and password file <none>
I20260923 15:05:36.663120 30814 fs_manager.cc:359] Metadata directory not provided
I20260923 15:05:36.663194 30814 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:05:36.663353 30814 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:05:36.664467 30814 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175691491353-13614-0/minicluster-data/ts-1/data/instance:
uuid: "adc38edd11e44836bf669600916de08e"
format_stamp: "Formatted at 2026-09-23 15:05:36 on dist-test-slave-cnb0"
server_key: "5f3a216fad673dd67cc7efe654836b96"
server_key_iv: "734525b90487833e97968bf11af30720"
server_key_version: "encryptionkey@0"
I20260923 15:05:36.664844 30814 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175691491353-13614-0/minicluster-data/ts-1/wal/instance:
uuid: "adc38edd11e44836bf669600916de08e"
format_stamp: "Formatted at 2026-09-23 15:05:36 on dist-test-slave-cnb0"
server_key: "5f3a216fad673dd67cc7efe654836b96"
server_key_iv: "734525b90487833e97968bf11af30720"
server_key_version: "encryptionkey@0"
I20260923 15:05:36.666208 30814 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 15:05:36.667027 30829 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:05:36.667299 30814 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:05:36.667416 30814 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175691491353-13614-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175691491353-13614-0/minicluster-data/ts-1/wal
uuid: "adc38edd11e44836bf669600916de08e"
format_stamp: "Formatted at 2026-09-23 15:05:36 on dist-test-slave-cnb0"
server_key: "5f3a216fad673dd67cc7efe654836b96"
server_key_iv: "734525b90487833e97968bf11af30720"
server_key_version: "encryptionkey@0"
I20260923 15:05:36.667513 30814 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175691491353-13614-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175691491353-13614-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175691491353-13614-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:05:36.698552 30814 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:05:36.700847 30814 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:05:36.701027 30814 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:05:36.701277 30814 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 15:05:36.701630 30814 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 15:05:36.701691 30814 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 15:05:36.701733 30814 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 15:05:36.701761 30814 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 15:05:36.707885 30814 rpc_server.cc:307] RPC server started. Bound to: 127.13.75.130:36715
I20260923 15:05:36.707934 30942 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.75.130:36715 every 8 connection(s)
I20260923 15:05:36.708304 30814 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175691491353-13614-0/minicluster-data/ts-1/data/info.pb
I20260923 15:05:36.709515 13614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKC4soy/build/release/bin/kudu as pid 30814
I20260923 15:05:36.709591 13614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175691491353-13614-0/minicluster-data/ts-1/wal/instance
I20260923 15:05:36.709802 13614 external_mini_cluster.cc:1428] Setting key 75100b45874d17fc56edc5cc7ea941bc
Sep 23 15:05:36 dist-test-slave-cnb0 krb5kdc[30573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175936, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.130@KRBTEST.COM for kudu/127.13.75.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.13.75.131@KRBTEST.COM; defaulting to no policy
I20260923 15:05:36.718549 30676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:36.709129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.130:56957 (local address 127.13.75.190:41925)
0923 15:05:36.709191 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0923 15:05:36.709193 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:05:36.709396 (+ 203us) server_negotiation.cc:407] Connection header received
0923 15:05:36.710042 (+ 646us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:05:36.710043 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:05:36.710060 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:05:36.710108 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:05:36.711193 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:05:36.711686 (+ 493us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:05:36.712856 (+ 1170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:05:36.713060 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:05:36.713148 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:05:36.715593 (+ 2445us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:05:36.715595 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:05:36.715595 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:05:36.715603 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:05:36.717044 (+ 1441us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:05:36.717509 (+ 465us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:05:36.717510 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:05:36.717511 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:05:36.717572 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:05:36.717844 (+ 272us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:05:36.717845 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:05:36.717845 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:05:36.717985 (+ 140us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:05:36.718065 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0923 15:05:36.718358 (+ 293us) server_negotiation.cc:299] Negotiation successful
0923 15:05:36.718453 (+ 95us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260923 15:05:36.718789 30945 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:36.709085 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:41925 (local address 127.13.75.130:56957)
0923 15:05:36.709269 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0923 15:05:36.709314 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0923 15:05:36.709928 (+ 614us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:05:36.710190 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:05:36.710194 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:05:36.710800 (+ 606us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:05:36.711088 (+ 288us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:05:36.711094 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:05:36.711790 (+ 696us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:05:36.711791 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:05:36.712754 (+ 963us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:05:36.712756 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:05:36.713158 (+ 402us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:05:36.713159 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:05:36.713296 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:05:36.713987 (+ 691us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:05:36.713995 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:05:36.715484 (+ 1489us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:05:36.717151 (+ 1667us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:05:36.717160 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:05:36.717162 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:05:36.717413 (+ 251us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:05:36.717643 (+ 230us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:05:36.717644 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:05:36.717644 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:05:36.717771 (+ 127us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:05:36.718055 (+ 284us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:05:36.718063 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:05:36.718216 (+ 153us) client_negotiation.cc:769] Sending connection context
0923 15:05:36.718336 (+ 120us) client_negotiation.cc:240] Negotiation successful
0923 15:05:36.718512 (+ 176us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":140,"thread_start_us":108,"threads_started":1}
I20260923 15:05:36.719319 30943 heartbeater.cc:347] Connected to a master server at 127.13.75.190:41925
I20260923 15:05:36.719455 30943 heartbeater.cc:464] Registering TS with master...
I20260923 15:05:36.719645 30943 heartbeater.cc:511] Master 127.13.75.190:41925 requested a full tablet report, sending...
I20260923 15:05:36.720037 30618 ts_manager.cc:194] Registered new tserver with Master: adc38edd11e44836bf669600916de08e (127.13.75.130:36715)
I20260923 15:05:36.720500 30618 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.75.130@KRBTEST.COM'} at 127.13.75.130:56957
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.75.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.75.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.131.keytab.
Entry for principal kudu/127.13.75.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.131.keytab.
I20260923 15:05:36.738996 13614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKC4soy/build/release/bin/kudu
/tmp/dist-test-taskKC4soy/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175691491353-13614-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175691491353-13614-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175691491353-13614-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175691491353-13614-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.75.131:0
--local_ip_for_outbound_sockets=127.13.75.131
--webserver_interface=127.13.75.131
--webserver_port=0
--tserver_master_addrs=127.13.75.190:41925
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.131.keytab
--principal=kudu/127.13.75.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.75.148:39491
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175691491353-13614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175691491353-13614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175691491353-13614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:05:36.824262 30948 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:05:36.824477 30948 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:05:36.824503 30948 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:05:36.825629 30948 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:05:36.825670 30948 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:05:36.825697 30948 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.75.131
I20260923 15:05:36.827230 30948 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.75.148:39491
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175691491353-13614-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175691491353-13614-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.131.keytab
--principal=kudu/127.13.75.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.75.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175691491353-13614-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.75.131
--webserver_port=0
--tserver_master_addrs=127.13.75.190:41925
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.30948
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.75.131
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175691491353-13614-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:05:36.827430 30948 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:05:36.827610 30948 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:05:36.832897 30953 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:05:36.832952 30956 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:05:36.832963 30948 server_base.cc:1034] running on GCE node
W20260923 15:05:36.833295 30954 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:05:36.833599 30948 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:05:36.833889 30948 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:05:36.835052 30948 hybrid_clock.cc:648] HybridClock initialized: now 1790175936834995 us; error 78 us; skew 500 ppm
Sep 23 15:05:36 dist-test-slave-cnb0 krb5kdc[30573](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175936, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:05:36.837672 30948 init.cc:377] Logged in from keytab as kudu/127.13.75.131@KRBTEST.COM (short username kudu)
I20260923 15:05:36.838076 30948 webserver.cc:460] Webserver started at http://127.13.75.131:36803/ using document root <none> and password file <none>
I20260923 15:05:36.838266 30948 fs_manager.cc:359] Metadata directory not provided
I20260923 15:05:36.838312 30948 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:05:36.838404 30948 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:05:36.839596 30948 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175691491353-13614-0/minicluster-data/ts-2/data/instance:
uuid: "f5a76bf04416427398c1e2d014f8ddf0"
format_stamp: "Formatted at 2026-09-23 15:05:36 on dist-test-slave-cnb0"
server_key: "3ead5b8951207b7a2678fdcb0c263f68"
server_key_iv: "b134186f80d9b4b55d4e4644fad63020"
server_key_version: "encryptionkey@0"
I20260923 15:05:36.840057 30948 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175691491353-13614-0/minicluster-data/ts-2/wal/instance:
uuid: "f5a76bf04416427398c1e2d014f8ddf0"
format_stamp: "Formatted at 2026-09-23 15:05:36 on dist-test-slave-cnb0"
server_key: "3ead5b8951207b7a2678fdcb0c263f68"
server_key_iv: "b134186f80d9b4b55d4e4644fad63020"
server_key_version: "encryptionkey@0"
I20260923 15:05:36.841492 30948 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 15:05:36.842357 30963 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:05:36.842581 30948 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:05:36.842702 30948 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175691491353-13614-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175691491353-13614-0/minicluster-data/ts-2/wal
uuid: "f5a76bf04416427398c1e2d014f8ddf0"
format_stamp: "Formatted at 2026-09-23 15:05:36 on dist-test-slave-cnb0"
server_key: "3ead5b8951207b7a2678fdcb0c263f68"
server_key_iv: "b134186f80d9b4b55d4e4644fad63020"
server_key_version: "encryptionkey@0"
I20260923 15:05:36.842764 30948 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175691491353-13614-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175691491353-13614-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175691491353-13614-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:05:36.855974 30948 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:05:36.858206 30948 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:05:36.858335 30948 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:05:36.858537 30948 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 15:05:36.858829 30948 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 15:05:36.858863 30948 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 15:05:36.858958 30948 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 15:05:36.858994 30948 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 15:05:36.865533 30948 rpc_server.cc:307] RPC server started. Bound to: 127.13.75.131:33727
I20260923 15:05:36.865761 31076 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.75.131:33727 every 8 connection(s)
I20260923 15:05:36.865963 30948 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175691491353-13614-0/minicluster-data/ts-2/data/info.pb
Sep 23 15:05:36 dist-test-slave-cnb0 krb5kdc[30573](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175936, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.131@KRBTEST.COM for kudu/127.13.75.190@KRBTEST.COM
I20260923 15:05:36.874408 13614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKC4soy/build/release/bin/kudu as pid 30948
I20260923 15:05:36.874511 13614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175691491353-13614-0/minicluster-data/ts-2/wal/instance
I20260923 15:05:36.874694 13614 external_mini_cluster.cc:1428] Setting key 148771a37b0a51500c52d7e1260c1542
I20260923 15:05:36.876372 30676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:36.866829 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.131:40657 (local address 127.13.75.190:41925)
0923 15:05:36.867021 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0923 15:05:36.867024 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:05:36.867186 (+ 162us) server_negotiation.cc:407] Connection header received
0923 15:05:36.867715 (+ 529us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:05:36.867717 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:05:36.867729 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:05:36.867777 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:05:36.868742 (+ 965us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:05:36.869299 (+ 557us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:05:36.870337 (+ 1038us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:05:36.870565 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:05:36.870661 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:05:36.873212 (+ 2551us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:05:36.873214 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:05:36.873215 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:05:36.873224 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:05:36.874630 (+ 1406us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:05:36.875230 (+ 600us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:05:36.875231 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:05:36.875232 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:05:36.875308 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:05:36.875619 (+ 311us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:05:36.875620 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:05:36.875620 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:05:36.875750 (+ 130us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:05:36.875863 (+ 113us) server_negotiation.cc:1035] Waiting for connection context
0923 15:05:36.876171 (+ 308us) server_negotiation.cc:299] Negotiation successful
0923 15:05:36.876276 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":158}
I20260923 15:05:36.876579 31079 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:36.866734 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:41925 (local address 127.13.75.131:40657)
0923 15:05:36.867044 (+ 310us) negotiation.cc:107] Waiting for socket to connect
0923 15:05:36.867091 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0923 15:05:36.867610 (+ 519us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:05:36.867846 (+ 236us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:05:36.867851 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:05:36.868439 (+ 588us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:05:36.868638 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:05:36.868642 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:05:36.869414 (+ 772us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:05:36.869415 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:05:36.870227 (+ 812us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:05:36.870229 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:05:36.870685 (+ 456us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:05:36.870686 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:05:36.870827 (+ 141us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:05:36.871559 (+ 732us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:05:36.871568 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:05:36.873111 (+ 1543us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:05:36.874752 (+ 1641us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:05:36.874760 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:05:36.874762 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:05:36.875146 (+ 384us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:05:36.875389 (+ 243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:05:36.875390 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:05:36.875390 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:05:36.875534 (+ 144us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:05:36.875854 (+ 320us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:05:36.875861 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:05:36.876016 (+ 155us) client_negotiation.cc:769] Sending connection context
0923 15:05:36.876129 (+ 113us) client_negotiation.cc:240] Negotiation successful
0923 15:05:36.876317 (+ 188us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":259,"thread_start_us":101,"threads_started":1}
I20260923 15:05:36.877076 31077 heartbeater.cc:347] Connected to a master server at 127.13.75.190:41925
I20260923 15:05:36.877187 31077 heartbeater.cc:464] Registering TS with master...
I20260923 15:05:36.877381 31077 heartbeater.cc:511] Master 127.13.75.190:41925 requested a full tablet report, sending...
I20260923 15:05:36.877746 30618 ts_manager.cc:194] Registered new tserver with Master: f5a76bf04416427398c1e2d014f8ddf0 (127.13.75.131:33727)
I20260923 15:05:36.878079 30618 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.75.131@KRBTEST.COM'} at 127.13.75.131:40657
I20260923 15:05:36.886510 13614 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260923 15:05:36.894454 30676 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:36.887960 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34648 (local address 127.13.75.190:41925)
0923 15:05:36.888046 (+ 86us) server_negotiation.cc:206] Beginning negotiation
0923 15:05:36.888048 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:05:36.888195 (+ 147us) server_negotiation.cc:407] Connection header received
0923 15:05:36.888222 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:05:36.888223 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:05:36.888234 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:05:36.888294 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:05:36.888906 (+ 612us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:05:36.889453 (+ 547us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:05:36.890353 (+ 900us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:05:36.890540 (+ 187us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:05:36.890597 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:05:36.891852 (+ 1255us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:05:36.891853 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:05:36.891854 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:05:36.891861 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:05:36.893330 (+ 1469us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:05:36.893656 (+ 326us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:05:36.893657 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:05:36.893657 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:05:36.893713 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:05:36.893951 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:05:36.893952 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:05:36.893952 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:05:36.894108 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:05:36.894186 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0923 15:05:36.894283 (+ 97us) server_negotiation.cc:299] Negotiation successful
0923 15:05:36.894349 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260923 15:05:36.896214 30618 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34648:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 15:05:36.897822 30618 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 15:05:36.905148 31088 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:36.900468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.131:33727 (local address 127.0.0.1:53204)
0923 15:05:36.900866 (+ 398us) negotiation.cc:107] Waiting for socket to connect
0923 15:05:36.900914 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0923 15:05:36.900987 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:05:36.901291 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:05:36.901295 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:05:36.901298 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:05:36.901402 (+ 104us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:05:36.901404 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:05:36.902932 (+ 1528us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:05:36.902933 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:05:36.903873 (+ 940us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:05:36.903876 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:05:36.904823 (+ 947us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:05:36.904824 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:05:36.904885 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:05:36.904896 (+ 11us) client_negotiation.cc:769] Sending connection context
0923 15:05:36.904992 (+ 96us) client_negotiation.cc:240] Negotiation successful
0923 15:05:36.905056 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":367,"mutex_wait_us":29,"thread_start_us":65,"threads_started":1}
I20260923 15:05:36.905157 31092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:36.900656 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53204 (local address 127.13.75.131:33727)
0923 15:05:36.900868 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0923 15:05:36.900873 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 15:05:36.900997 (+ 124us) server_negotiation.cc:407] Connection header received
0923 15:05:36.901083 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:05:36.901086 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:05:36.901104 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:05:36.901195 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:05:36.901490 (+ 295us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:05:36.902789 (+ 1299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:05:36.903980 (+ 1191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:05:36.904728 (+ 748us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:05:36.904809 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:05:36.904873 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0923 15:05:36.905022 (+ 149us) server_negotiation.cc:299] Negotiation successful
0923 15:05:36.905068 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":174,"thread_start_us":130,"threads_started":1}
I20260923 15:05:36.905243 31091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:36.900656 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:50502 (local address 127.13.75.130:36715)
0923 15:05:36.900957 (+ 301us) server_negotiation.cc:206] Beginning negotiation
0923 15:05:36.900960 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:05:36.901270 (+ 310us) server_negotiation.cc:407] Connection header received
0923 15:05:36.901319 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:05:36.901322 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:05:36.901338 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:05:36.901409 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:05:36.901651 (+ 242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:05:36.902504 (+ 853us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:05:36.904137 (+ 1633us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:05:36.904875 (+ 738us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:05:36.904945 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:05:36.905007 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0923 15:05:36.905115 (+ 108us) server_negotiation.cc:299] Negotiation successful
0923 15:05:36.905158 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":263,"thread_start_us":102,"threads_started":1}
I20260923 15:05:36.905454 31089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:36.900468 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.130:36715 (local address 127.0.0.1:50502)
0923 15:05:36.901093 (+ 625us) negotiation.cc:107] Waiting for socket to connect
0923 15:05:36.901101 (+ 8us) client_negotiation.cc:174] Beginning negotiation
0923 15:05:36.901147 (+ 46us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:05:36.901521 (+ 374us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:05:36.901522 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:05:36.901523 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:05:36.901568 (+ 45us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:05:36.901569 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:05:36.902653 (+ 1084us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:05:36.902655 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:05:36.904044 (+ 1389us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:05:36.904048 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:05:36.904953 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:05:36.904954 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:05:36.905012 (+ 58us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:05:36.905017 (+ 5us) client_negotiation.cc:769] Sending connection context
0923 15:05:36.905338 (+ 321us) client_negotiation.cc:240] Negotiation successful
0923 15:05:36.905388 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":610,"mutex_wait_us":9,"thread_start_us":96,"threads_started":1}
I20260923 15:05:36.906028 30877 tablet_service.cc:1467] Processing CreateTablet for tablet d3d07e147015469f8755d5835ab82aef (DEFAULT_TABLE table=test-table [id=4035f55947f94050bda7a7c8ff6b2373]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 15:05:36.906028 31011 tablet_service.cc:1467] Processing CreateTablet for tablet d3d07e147015469f8755d5835ab82aef (DEFAULT_TABLE table=test-table [id=4035f55947f94050bda7a7c8ff6b2373]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 15:05:36.906400 31011 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d3d07e147015469f8755d5835ab82aef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:05:36.906400 30877 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d3d07e147015469f8755d5835ab82aef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:05:36.906558 31090 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:36.900469 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.129:43387 (local address 127.0.0.1:40884)
0923 15:05:36.900879 (+ 410us) negotiation.cc:107] Waiting for socket to connect
0923 15:05:36.900914 (+ 35us) client_negotiation.cc:174] Beginning negotiation
0923 15:05:36.900987 (+ 73us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:05:36.901291 (+ 304us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:05:36.901295 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:05:36.901299 (+ 4us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:05:36.901402 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:05:36.901405 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:05:36.902914 (+ 1509us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:05:36.902915 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:05:36.904058 (+ 1143us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:05:36.904060 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:05:36.906293 (+ 2233us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:05:36.906294 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:05:36.906355 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:05:36.906358 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 15:05:36.906403 (+ 45us) client_negotiation.cc:240] Negotiation successful
0923 15:05:36.906486 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":392,"mutex_wait_us":47,"thread_start_us":110,"threads_started":1}
I20260923 15:05:36.906577 31093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:36.900717 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40884 (local address 127.13.75.129:43387)
0923 15:05:36.901057 (+ 340us) server_negotiation.cc:206] Beginning negotiation
0923 15:05:36.901061 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 15:05:36.901070 (+ 9us) server_negotiation.cc:407] Connection header received
0923 15:05:36.901106 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:05:36.901108 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:05:36.901121 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:05:36.901194 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:05:36.901491 (+ 297us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:05:36.902775 (+ 1284us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:05:36.905046 (+ 2271us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:05:36.906162 (+ 1116us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:05:36.906330 (+ 168us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:05:36.906394 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0923 15:05:36.906420 (+ 26us) server_negotiation.cc:299] Negotiation successful
0923 15:05:36.906458 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":304,"thread_start_us":114,"threads_started":1}
I20260923 15:05:36.907305 30743 tablet_service.cc:1467] Processing CreateTablet for tablet d3d07e147015469f8755d5835ab82aef (DEFAULT_TABLE table=test-table [id=4035f55947f94050bda7a7c8ff6b2373]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 15:05:36.907649 30743 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet d3d07e147015469f8755d5835ab82aef. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:05:36.910043 31095 tablet_bootstrap.cc:492] T d3d07e147015469f8755d5835ab82aef P adc38edd11e44836bf669600916de08e: Bootstrap starting.
I20260923 15:05:36.910547 31096 tablet_bootstrap.cc:492] T d3d07e147015469f8755d5835ab82aef P 2b7a2526d96644fba233866d7768451d: Bootstrap starting.
I20260923 15:05:36.910857 31094 tablet_bootstrap.cc:492] T d3d07e147015469f8755d5835ab82aef P f5a76bf04416427398c1e2d014f8ddf0: Bootstrap starting.
I20260923 15:05:36.911262 31096 tablet_bootstrap.cc:654] T d3d07e147015469f8755d5835ab82aef P 2b7a2526d96644fba233866d7768451d: Neither blocks nor log segments found. Creating new log.
I20260923 15:05:36.911547 31094 tablet_bootstrap.cc:654] T d3d07e147015469f8755d5835ab82aef P f5a76bf04416427398c1e2d014f8ddf0: Neither blocks nor log segments found. Creating new log.
I20260923 15:05:36.911947 31096 log.cc:824] T d3d07e147015469f8755d5835ab82aef P 2b7a2526d96644fba233866d7768451d: Log is configured to *not* fsync() on all Append() calls
I20260923 15:05:36.912192 31094 log.cc:824] T d3d07e147015469f8755d5835ab82aef P f5a76bf04416427398c1e2d014f8ddf0: Log is configured to *not* fsync() on all Append() calls
I20260923 15:05:36.912657 31096 tablet_bootstrap.cc:492] T d3d07e147015469f8755d5835ab82aef P 2b7a2526d96644fba233866d7768451d: No bootstrap required, opened a new log
I20260923 15:05:36.912720 31095 tablet_bootstrap.cc:654] T d3d07e147015469f8755d5835ab82aef P adc38edd11e44836bf669600916de08e: Neither blocks nor log segments found. Creating new log.
I20260923 15:05:36.912750 31096 ts_tablet_manager.cc:1397] T d3d07e147015469f8755d5835ab82aef P 2b7a2526d96644fba233866d7768451d: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260923 15:05:36.913111 31094 tablet_bootstrap.cc:492] T d3d07e147015469f8755d5835ab82aef P f5a76bf04416427398c1e2d014f8ddf0: No bootstrap required, opened a new log
I20260923 15:05:36.913178 31094 ts_tablet_manager.cc:1397] T d3d07e147015469f8755d5835ab82aef P f5a76bf04416427398c1e2d014f8ddf0: Time spent bootstrapping tablet: real 0.002s user 0.000s sys 0.002s
I20260923 15:05:36.913311 31095 log.cc:824] T d3d07e147015469f8755d5835ab82aef P adc38edd11e44836bf669600916de08e: Log is configured to *not* fsync() on all Append() calls
I20260923 15:05:36.913972 31095 tablet_bootstrap.cc:492] T d3d07e147015469f8755d5835ab82aef P adc38edd11e44836bf669600916de08e: No bootstrap required, opened a new log
I20260923 15:05:36.914052 31095 ts_tablet_manager.cc:1397] T d3d07e147015469f8755d5835ab82aef P adc38edd11e44836bf669600916de08e: Time spent bootstrapping tablet: real 0.004s user 0.002s sys 0.000s
I20260923 15:05:36.914914 31096 raft_consensus.cc:348] T d3d07e147015469f8755d5835ab82aef P 2b7a2526d96644fba233866d7768451d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5a76bf04416427398c1e2d014f8ddf0" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 33727 } } peers { permanent_uuid: "2b7a2526d96644fba233866d7768451d" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 43387 } } peers { permanent_uuid: "adc38edd11e44836bf669600916de08e" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 36715 } }
I20260923 15:05:36.915105 31096 raft_consensus.cc:374] T d3d07e147015469f8755d5835ab82aef P 2b7a2526d96644fba233866d7768451d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:05:36.915143 31096 raft_consensus.cc:729] T d3d07e147015469f8755d5835ab82aef P 2b7a2526d96644fba233866d7768451d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2b7a2526d96644fba233866d7768451d, State: Initialized, Role: FOLLOWER
I20260923 15:05:36.915277 31096 consensus_queue.cc:260] T d3d07e147015469f8755d5835ab82aef P 2b7a2526d96644fba233866d7768451d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5a76bf04416427398c1e2d014f8ddf0" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 33727 } } peers { permanent_uuid: "2b7a2526d96644fba233866d7768451d" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 43387 } } peers { permanent_uuid: "adc38edd11e44836bf669600916de08e" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 36715 } }
I20260923 15:05:36.915387 31094 raft_consensus.cc:348] T d3d07e147015469f8755d5835ab82aef P f5a76bf04416427398c1e2d014f8ddf0 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5a76bf04416427398c1e2d014f8ddf0" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 33727 } } peers { permanent_uuid: "2b7a2526d96644fba233866d7768451d" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 43387 } } peers { permanent_uuid: "adc38edd11e44836bf669600916de08e" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 36715 } }
I20260923 15:05:36.915563 31094 raft_consensus.cc:374] T d3d07e147015469f8755d5835ab82aef P f5a76bf04416427398c1e2d014f8ddf0 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:05:36.915575 31096 ts_tablet_manager.cc:1428] T d3d07e147015469f8755d5835ab82aef P 2b7a2526d96644fba233866d7768451d: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260923 15:05:36.915611 31094 raft_consensus.cc:729] T d3d07e147015469f8755d5835ab82aef P f5a76bf04416427398c1e2d014f8ddf0 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: f5a76bf04416427398c1e2d014f8ddf0, State: Initialized, Role: FOLLOWER
I20260923 15:05:36.915690 30809 heartbeater.cc:503] Master 127.13.75.190:41925 was elected leader, sending a full tablet report...
I20260923 15:05:36.915748 31094 consensus_queue.cc:260] T d3d07e147015469f8755d5835ab82aef P f5a76bf04416427398c1e2d014f8ddf0 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5a76bf04416427398c1e2d014f8ddf0" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 33727 } } peers { permanent_uuid: "2b7a2526d96644fba233866d7768451d" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 43387 } } peers { permanent_uuid: "adc38edd11e44836bf669600916de08e" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 36715 } }
I20260923 15:05:36.916045 31094 ts_tablet_manager.cc:1428] T d3d07e147015469f8755d5835ab82aef P f5a76bf04416427398c1e2d014f8ddf0: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260923 15:05:36.916097 31077 heartbeater.cc:503] Master 127.13.75.190:41925 was elected leader, sending a full tablet report...
I20260923 15:05:36.916088 31095 raft_consensus.cc:348] T d3d07e147015469f8755d5835ab82aef P adc38edd11e44836bf669600916de08e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5a76bf04416427398c1e2d014f8ddf0" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 33727 } } peers { permanent_uuid: "2b7a2526d96644fba233866d7768451d" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 43387 } } peers { permanent_uuid: "adc38edd11e44836bf669600916de08e" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 36715 } }
I20260923 15:05:36.916251 31095 raft_consensus.cc:374] T d3d07e147015469f8755d5835ab82aef P adc38edd11e44836bf669600916de08e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:05:36.916311 31095 raft_consensus.cc:729] T d3d07e147015469f8755d5835ab82aef P adc38edd11e44836bf669600916de08e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: adc38edd11e44836bf669600916de08e, State: Initialized, Role: FOLLOWER
I20260923 15:05:36.916419 31095 consensus_queue.cc:260] T d3d07e147015469f8755d5835ab82aef P adc38edd11e44836bf669600916de08e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5a76bf04416427398c1e2d014f8ddf0" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 33727 } } peers { permanent_uuid: "2b7a2526d96644fba233866d7768451d" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 43387 } } peers { permanent_uuid: "adc38edd11e44836bf669600916de08e" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 36715 } }
I20260923 15:05:36.916685 31095 ts_tablet_manager.cc:1428] T d3d07e147015469f8755d5835ab82aef P adc38edd11e44836bf669600916de08e: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260923 15:05:36.916857 30943 heartbeater.cc:503] Master 127.13.75.190:41925 was elected leader, sending a full tablet report...
W20260923 15:05:36.958940 30944 tablet.cc:2369] T d3d07e147015469f8755d5835ab82aef P adc38edd11e44836bf669600916de08e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 15:05:37.000928 31100 raft_consensus.cc:482] T d3d07e147015469f8755d5835ab82aef P 2b7a2526d96644fba233866d7768451d [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 15:05:37.001062 31100 raft_consensus.cc:504] T d3d07e147015469f8755d5835ab82aef P 2b7a2526d96644fba233866d7768451d [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5a76bf04416427398c1e2d014f8ddf0" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 33727 } } peers { permanent_uuid: "2b7a2526d96644fba233866d7768451d" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 43387 } } peers { permanent_uuid: "adc38edd11e44836bf669600916de08e" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 36715 } }
I20260923 15:05:37.001433 31100 leader_election.cc:288] T d3d07e147015469f8755d5835ab82aef P 2b7a2526d96644fba233866d7768451d [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers f5a76bf04416427398c1e2d014f8ddf0 (127.13.75.131:33727), adc38edd11e44836bf669600916de08e (127.13.75.130:36715)
I20260923 15:05:37.004823 31092 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:37.001863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.129:41613 (local address 127.13.75.131:33727)
0923 15:05:37.001915 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0923 15:05:37.001917 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:05:37.001928 (+ 11us) server_negotiation.cc:407] Connection header received
0923 15:05:37.001957 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:05:37.001958 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:05:37.001968 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:05:37.002026 (+ 58us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:05:37.002269 (+ 243us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:05:37.002902 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:05:37.003968 (+ 1066us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:05:37.004478 (+ 510us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:05:37.004536 (+ 58us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:05:37.004543 (+ 7us) server_negotiation.cc:1035] Waiting for connection context
0923 15:05:37.004720 (+ 177us) server_negotiation.cc:299] Negotiation successful
0923 15:05:37.004767 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260923 15:05:37.004877 30811 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:37.001661 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.131:33727 (local address 127.13.75.129:41613)
0923 15:05:37.001760 (+ 99us) negotiation.cc:107] Waiting for socket to connect
0923 15:05:37.001773 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0923 15:05:37.001827 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:05:37.002083 (+ 256us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:05:37.002085 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:05:37.002087 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:05:37.002171 (+ 84us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:05:37.002173 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:05:37.003004 (+ 831us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:05:37.003005 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:05:37.003863 (+ 858us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:05:37.003866 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:05:37.004553 (+ 687us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:05:37.004555 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:05:37.004643 (+ 88us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:05:37.004647 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 15:05:37.004707 (+ 60us) client_negotiation.cc:240] Negotiation successful
0923 15:05:37.004751 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":74}
I20260923 15:05:37.005239 31103 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:37.001702 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.130:36715 (local address 127.13.75.129:36179)
0923 15:05:37.001898 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0923 15:05:37.001907 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 15:05:37.001951 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:05:37.002257 (+ 306us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:05:37.002267 (+ 10us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:05:37.002269 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:05:37.002420 (+ 151us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:05:37.002422 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:05:37.003233 (+ 811us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:05:37.003234 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:05:37.004229 (+ 995us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:05:37.004232 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:05:37.004981 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:05:37.004982 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:05:37.005069 (+ 87us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:05:37.005073 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 15:05:37.005128 (+ 55us) client_negotiation.cc:240] Negotiation successful
0923 15:05:37.005170 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":185,"thread_start_us":82,"threads_started":1}
I20260923 15:05:37.005291 31091 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:37.001883 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.129:36179 (local address 127.13.75.130:36715)
0923 15:05:37.001929 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0923 15:05:37.001931 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:05:37.001953 (+ 22us) server_negotiation.cc:407] Connection header received
0923 15:05:37.002087 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:05:37.002089 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:05:37.002100 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:05:37.002187 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:05:37.002520 (+ 333us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:05:37.003127 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:05:37.004338 (+ 1211us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:05:37.004865 (+ 527us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:05:37.004933 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:05:37.004943 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0923 15:05:37.005156 (+ 213us) server_negotiation.cc:299] Negotiation successful
0923 15:05:37.005196 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260923 15:05:37.005306 31031 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3d07e147015469f8755d5835ab82aef" candidate_uuid: "2b7a2526d96644fba233866d7768451d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5a76bf04416427398c1e2d014f8ddf0" is_pre_election: true
I20260923 15:05:37.005496 31031 raft_consensus.cc:2454] T d3d07e147015469f8755d5835ab82aef P f5a76bf04416427398c1e2d014f8ddf0 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2b7a2526d96644fba233866d7768451d in term 0.
I20260923 15:05:37.005640 30897 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3d07e147015469f8755d5835ab82aef" candidate_uuid: "2b7a2526d96644fba233866d7768451d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "adc38edd11e44836bf669600916de08e" is_pre_election: true
I20260923 15:05:37.005764 30698 leader_election.cc:302] T d3d07e147015469f8755d5835ab82aef P 2b7a2526d96644fba233866d7768451d [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b7a2526d96644fba233866d7768451d, f5a76bf04416427398c1e2d014f8ddf0; no voters:
I20260923 15:05:37.005805 30897 raft_consensus.cc:2454] T d3d07e147015469f8755d5835ab82aef P adc38edd11e44836bf669600916de08e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 2b7a2526d96644fba233866d7768451d in term 0.
I20260923 15:05:37.005937 31100 raft_consensus.cc:2781] T d3d07e147015469f8755d5835ab82aef P 2b7a2526d96644fba233866d7768451d [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 15:05:37.006013 31100 raft_consensus.cc:482] T d3d07e147015469f8755d5835ab82aef P 2b7a2526d96644fba233866d7768451d [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 15:05:37.006058 31100 raft_consensus.cc:3037] T d3d07e147015469f8755d5835ab82aef P 2b7a2526d96644fba233866d7768451d [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:05:37.006800 31100 raft_consensus.cc:504] T d3d07e147015469f8755d5835ab82aef P 2b7a2526d96644fba233866d7768451d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5a76bf04416427398c1e2d014f8ddf0" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 33727 } } peers { permanent_uuid: "2b7a2526d96644fba233866d7768451d" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 43387 } } peers { permanent_uuid: "adc38edd11e44836bf669600916de08e" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 36715 } }
I20260923 15:05:37.006992 31100 leader_election.cc:288] T d3d07e147015469f8755d5835ab82aef P 2b7a2526d96644fba233866d7768451d [CANDIDATE]: Term 1 election: Requested vote from peers f5a76bf04416427398c1e2d014f8ddf0 (127.13.75.131:33727), adc38edd11e44836bf669600916de08e (127.13.75.130:36715)
I20260923 15:05:37.007221 31031 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3d07e147015469f8755d5835ab82aef" candidate_uuid: "2b7a2526d96644fba233866d7768451d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "f5a76bf04416427398c1e2d014f8ddf0"
I20260923 15:05:37.007191 30897 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "d3d07e147015469f8755d5835ab82aef" candidate_uuid: "2b7a2526d96644fba233866d7768451d" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "adc38edd11e44836bf669600916de08e"
I20260923 15:05:37.007308 31031 raft_consensus.cc:3037] T d3d07e147015469f8755d5835ab82aef P f5a76bf04416427398c1e2d014f8ddf0 [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:05:37.007332 30897 raft_consensus.cc:3037] T d3d07e147015469f8755d5835ab82aef P adc38edd11e44836bf669600916de08e [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:05:37.008050 31031 raft_consensus.cc:2454] T d3d07e147015469f8755d5835ab82aef P f5a76bf04416427398c1e2d014f8ddf0 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2b7a2526d96644fba233866d7768451d in term 1.
I20260923 15:05:37.008132 30897 raft_consensus.cc:2454] T d3d07e147015469f8755d5835ab82aef P adc38edd11e44836bf669600916de08e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 2b7a2526d96644fba233866d7768451d in term 1.
I20260923 15:05:37.008214 30698 leader_election.cc:302] T d3d07e147015469f8755d5835ab82aef P 2b7a2526d96644fba233866d7768451d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2b7a2526d96644fba233866d7768451d, f5a76bf04416427398c1e2d014f8ddf0; no voters:
I20260923 15:05:37.008327 31100 raft_consensus.cc:2781] T d3d07e147015469f8755d5835ab82aef P 2b7a2526d96644fba233866d7768451d [term 1 FOLLOWER]: Leader election won for term 1
I20260923 15:05:37.008472 31100 raft_consensus.cc:686] T d3d07e147015469f8755d5835ab82aef P 2b7a2526d96644fba233866d7768451d [term 1 LEADER]: Becoming Leader. State: Replica: 2b7a2526d96644fba233866d7768451d, State: Running, Role: LEADER
W20260923 15:05:37.008524 30810 tablet.cc:2369] T d3d07e147015469f8755d5835ab82aef P 2b7a2526d96644fba233866d7768451d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 15:05:37.008742 31100 consensus_queue.cc:237] T d3d07e147015469f8755d5835ab82aef P 2b7a2526d96644fba233866d7768451d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5a76bf04416427398c1e2d014f8ddf0" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 33727 } } peers { permanent_uuid: "2b7a2526d96644fba233866d7768451d" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 43387 } } peers { permanent_uuid: "adc38edd11e44836bf669600916de08e" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 36715 } }
I20260923 15:05:37.009665 30617 catalog_manager.cc:5495] T d3d07e147015469f8755d5835ab82aef P 2b7a2526d96644fba233866d7768451d reported cstate change: term changed from 0 to 1, leader changed from <none> to 2b7a2526d96644fba233866d7768451d (127.13.75.129). New cstate: current_term: 1 leader_uuid: "2b7a2526d96644fba233866d7768451d" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "f5a76bf04416427398c1e2d014f8ddf0" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 33727 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2b7a2526d96644fba233866d7768451d" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 43387 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "adc38edd11e44836bf669600916de08e" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 36715 } health_report { overall_health: UNKNOWN } } }
I20260923 15:05:37.023394 31093 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:37.020830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40888 (local address 127.13.75.129:43387)
0923 15:05:37.020902 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0923 15:05:37.020904 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:05:37.020914 (+ 10us) server_negotiation.cc:407] Connection header received
0923 15:05:37.021032 (+ 118us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:05:37.021033 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:05:37.021056 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:05:37.021137 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 15:05:37.021280 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:05:37.021759 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:05:37.022623 (+ 864us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:05:37.022820 (+ 197us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:05:37.022930 (+ 110us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:05:37.023073 (+ 143us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 15:05:37.023177 (+ 104us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 15:05:37.023240 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0923 15:05:37.023296 (+ 56us) server_negotiation.cc:299] Negotiation successful
0923 15:05:37.023313 (+ 17us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260923 15:05:37.025285 30897 raft_consensus.cc:1261] T d3d07e147015469f8755d5835ab82aef P adc38edd11e44836bf669600916de08e [term 1 FOLLOWER]: Refusing update from remote peer 2b7a2526d96644fba233866d7768451d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 15:05:37.025318 31031 raft_consensus.cc:1261] T d3d07e147015469f8755d5835ab82aef P f5a76bf04416427398c1e2d014f8ddf0 [term 1 FOLLOWER]: Refusing update from remote peer 2b7a2526d96644fba233866d7768451d: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 15:05:37.025698 31104 consensus_queue.cc:1035] T d3d07e147015469f8755d5835ab82aef P 2b7a2526d96644fba233866d7768451d [LEADER]: Connected to new peer: Peer: permanent_uuid: "adc38edd11e44836bf669600916de08e" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 36715 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 15:05:37.025825 31100 consensus_queue.cc:1035] T d3d07e147015469f8755d5835ab82aef P 2b7a2526d96644fba233866d7768451d [LEADER]: Connected to new peer: Peer: permanent_uuid: "f5a76bf04416427398c1e2d014f8ddf0" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 33727 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 15:05:37.028127 31107 mvcc.cc:205] Tried to move back new op lower bound from 7332560638052798464 to 7332560637988573184. Current Snapshot: MvccSnapshot[applied={T|T < 7332560638052798464}]
I20260923 15:05:37.028693 31109 mvcc.cc:205] Tried to move back new op lower bound from 7332560638052798464 to 7332560637988573184. Current Snapshot: MvccSnapshot[applied={T|T < 7332560638052798464}]
I20260923 15:05:37.029057 31108 mvcc.cc:205] Tried to move back new op lower bound from 7332560638052798464 to 7332560637988573184. Current Snapshot: MvccSnapshot[applied={T|T < 7332560638052798464}]
I20260923 15:05:37.030763 30618 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34648:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 15:05:37.030908 30618 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:34648:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 15:05:37.031774 30618 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9fd42eea14af4c2b9e47ec30f042b9af: Sending DeleteTablet for 3 replicas of tablet d3d07e147015469f8755d5835ab82aef
I20260923 15:05:37.032114 31011 tablet_service.cc:1514] Processing DeleteTablet for tablet d3d07e147015469f8755d5835ab82aef with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 15:05:37 UTC) from {username='kudu', principal='kudu/127.13.75.190@KRBTEST.COM'} at 127.0.0.1:53204
I20260923 15:05:37.032157 30743 tablet_service.cc:1514] Processing DeleteTablet for tablet d3d07e147015469f8755d5835ab82aef with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 15:05:37 UTC) from {username='kudu', principal='kudu/127.13.75.190@KRBTEST.COM'} at 127.0.0.1:40884
I20260923 15:05:37.032434 13614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKC4soy/build/release/bin/kudu with pid 30680
I20260923 15:05:37.032474 30877 tablet_service.cc:1514] Processing DeleteTablet for tablet d3d07e147015469f8755d5835ab82aef with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 15:05:37 UTC) from {username='kudu', principal='kudu/127.13.75.190@KRBTEST.COM'} at 127.0.0.1:50502
I20260923 15:05:37.032542 31117 tablet_replica.cc:330] T d3d07e147015469f8755d5835ab82aef P 2b7a2526d96644fba233866d7768451d: stopping tablet replica
I20260923 15:05:37.032653 31117 raft_consensus.cc:2229] T d3d07e147015469f8755d5835ab82aef P 2b7a2526d96644fba233866d7768451d [term 1 LEADER]: Raft consensus shutting down.
I20260923 15:05:37.032743 31117 raft_consensus.cc:2258] T d3d07e147015469f8755d5835ab82aef P 2b7a2526d96644fba233866d7768451d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 15:05:37.033131 31118 tablet_replica.cc:330] T d3d07e147015469f8755d5835ab82aef P adc38edd11e44836bf669600916de08e: stopping tablet replica
I20260923 15:05:37.033229 31118 raft_consensus.cc:2229] T d3d07e147015469f8755d5835ab82aef P adc38edd11e44836bf669600916de08e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 15:05:37.033360 31118 raft_consensus.cc:2258] T d3d07e147015469f8755d5835ab82aef P adc38edd11e44836bf669600916de08e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 15:05:37.033540 31116 tablet_replica.cc:330] T d3d07e147015469f8755d5835ab82aef P f5a76bf04416427398c1e2d014f8ddf0: stopping tablet replica
I20260923 15:05:37.033627 31116 raft_consensus.cc:2229] T d3d07e147015469f8755d5835ab82aef P f5a76bf04416427398c1e2d014f8ddf0 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 15:05:37.033717 31116 raft_consensus.cc:2258] T d3d07e147015469f8755d5835ab82aef P f5a76bf04416427398c1e2d014f8ddf0 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 15:05:37.033849 31118 ts_tablet_manager.cc:1905] T d3d07e147015469f8755d5835ab82aef P adc38edd11e44836bf669600916de08e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 15:05:37.034132 31116 ts_tablet_manager.cc:1905] T d3d07e147015469f8755d5835ab82aef P f5a76bf04416427398c1e2d014f8ddf0: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 15:05:37.035439 31118 ts_tablet_manager.cc:1918] T d3d07e147015469f8755d5835ab82aef P adc38edd11e44836bf669600916de08e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 15:05:37.035511 31118 log.cc:1196] T d3d07e147015469f8755d5835ab82aef P adc38edd11e44836bf669600916de08e: Deleting WAL directory at /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175691491353-13614-0/minicluster-data/ts-1/wal/wals/d3d07e147015469f8755d5835ab82aef
I20260923 15:05:37.035847 31118 ts_tablet_manager.cc:1939] T d3d07e147015469f8755d5835ab82aef P adc38edd11e44836bf669600916de08e: Deleting consensus metadata
I20260923 15:05:37.036011 31116 ts_tablet_manager.cc:1918] T d3d07e147015469f8755d5835ab82aef P f5a76bf04416427398c1e2d014f8ddf0: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 15:05:37.036088 31116 log.cc:1196] T d3d07e147015469f8755d5835ab82aef P f5a76bf04416427398c1e2d014f8ddf0: Deleting WAL directory at /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_0.1790175691491353-13614-0/minicluster-data/ts-2/wal/wals/d3d07e147015469f8755d5835ab82aef
I20260923 15:05:37.036264 30603 catalog_manager.cc:4841] TS adc38edd11e44836bf669600916de08e (127.13.75.130:36715): tablet d3d07e147015469f8755d5835ab82aef (table test-table [id=4035f55947f94050bda7a7c8ff6b2373]) successfully deleted
I20260923 15:05:37.036482 31116 ts_tablet_manager.cc:1939] T d3d07e147015469f8755d5835ab82aef P f5a76bf04416427398c1e2d014f8ddf0: Deleting consensus metadata
I20260923 15:05:37.036900 30604 catalog_manager.cc:4841] TS f5a76bf04416427398c1e2d014f8ddf0 (127.13.75.131:33727): tablet d3d07e147015469f8755d5835ab82aef (table test-table [id=4035f55947f94050bda7a7c8ff6b2373]) successfully deleted
W20260923 15:05:37.039693 30602 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.75.129:43387 (error 108)
I20260923 15:05:37.040355 13614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKC4soy/build/release/bin/kudu with pid 30814
I20260923 15:05:37.040395 31089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:37.040176 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.129:43387 (local address 127.0.0.1:40904)
0923 15:05:37.040287 (+ 111us) negotiation.cc:107] Waiting for socket to connect
0923 15:05:37.040347 (+ 60us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.129:43387: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":81}
W20260923 15:05:37.043056 30602 catalog_manager.cc:4584] TS 2b7a2526d96644fba233866d7768451d (127.13.75.129:43387): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet d3d07e147015469f8755d5835ab82aef: Network error: Client connection negotiation failed: client connection to 127.13.75.129:43387: connect: Connection refused (error 111)
I20260923 15:05:37.046126 13614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKC4soy/build/release/bin/kudu with pid 30948
I20260923 15:05:37.050678 13614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKC4soy/build/release/bin/kudu with pid 30586
2026-09-23T15:05:37Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/0 (2078 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/1
2026-09-23T15:05:37Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T15:05:37Z Disabled control of system clock
I20260923 15:05:37.079080 13614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKC4soy/build/release/bin/kudu
/tmp/dist-test-taskKC4soy/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790175691491353-13614-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790175691491353-13614-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790175691491353-13614-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790175691491353-13614-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.75.190:43935
--webserver_interface=127.13.75.190
--webserver_port=0
--builtin_ntp_servers=127.13.75.148:34193
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.75.190:43935
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260923 15:05:37.161721 31123 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:05:37.161952 31123 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:05:37.161973 31123 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:05:37.163149 31123 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 15:05:37.163204 31123 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:05:37.163226 31123 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:05:37.163249 31123 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 15:05:37.163267 31123 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 15:05:37.164613 31123 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.75.148:34193
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790175691491353-13614-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790175691491353-13614-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.75.190:43935
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.75.190:43935
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790175691491353-13614-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.75.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31123
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790175691491353-13614-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:05:37.164815 31123 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:05:37.165066 31123 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:05:37.170547 31128 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:05:37.170620 31131 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:05:37.170547 31129 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:05:37.170604 31123 server_base.cc:1034] running on GCE node
I20260923 15:05:37.171164 31123 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:05:37.171540 31123 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:05:37.172713 31123 hybrid_clock.cc:648] HybridClock initialized: now 1790175937172694 us; error 56 us; skew 500 ppm
I20260923 15:05:37.174093 31123 webserver.cc:460] Webserver started at http://127.13.75.190:46319/ using document root <none> and password file <none>
I20260923 15:05:37.174373 31123 fs_manager.cc:359] Metadata directory not provided
I20260923 15:05:37.174448 31123 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:05:37.174602 31123 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:05:37.175783 31123 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790175691491353-13614-0/minicluster-data/master-0/data/instance:
uuid: "b7576b0ea7b44739b14360a89cff626e"
format_stamp: "Formatted at 2026-09-23 15:05:37 on dist-test-slave-cnb0"
server_key: "73ef951166a8e59bb4d4fddb88ffedae"
server_key_iv: "06a275482795d724e60fc19a594c7206"
server_key_version: "encryptionkey@0"
I20260923 15:05:37.176154 31123 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790175691491353-13614-0/minicluster-data/master-0/wal/instance:
uuid: "b7576b0ea7b44739b14360a89cff626e"
format_stamp: "Formatted at 2026-09-23 15:05:37 on dist-test-slave-cnb0"
server_key: "73ef951166a8e59bb4d4fddb88ffedae"
server_key_iv: "06a275482795d724e60fc19a594c7206"
server_key_version: "encryptionkey@0"
I20260923 15:05:37.177520 31123 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260923 15:05:37.178301 31137 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:05:37.178478 31123 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260923 15:05:37.178586 31123 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790175691491353-13614-0/minicluster-data/master-0/data,/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790175691491353-13614-0/minicluster-data/master-0/wal
uuid: "b7576b0ea7b44739b14360a89cff626e"
format_stamp: "Formatted at 2026-09-23 15:05:37 on dist-test-slave-cnb0"
server_key: "73ef951166a8e59bb4d4fddb88ffedae"
server_key_iv: "06a275482795d724e60fc19a594c7206"
server_key_version: "encryptionkey@0"
I20260923 15:05:37.178673 31123 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790175691491353-13614-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790175691491353-13614-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790175691491353-13614-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:05:37.197820 31123 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:05:37.198192 31123 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:05:37.198357 31123 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:05:37.202523 31123 rpc_server.cc:307] RPC server started. Bound to: 127.13.75.190:43935
I20260923 15:05:37.202561 31189 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.75.190:43935 every 8 connection(s)
I20260923 15:05:37.202932 31123 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790175691491353-13614-0/minicluster-data/master-0/data/info.pb
I20260923 15:05:37.203924 13614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKC4soy/build/release/bin/kudu as pid 31123
I20260923 15:05:37.204046 13614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790175691491353-13614-0/minicluster-data/master-0/wal/instance
I20260923 15:05:37.204154 31190 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:05:37.204340 13614 external_mini_cluster.cc:1428] Setting key 59c5bf3b4c82cfb19efed7f1a2d5c784
I20260923 15:05:37.208390 31190 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b7576b0ea7b44739b14360a89cff626e: Bootstrap starting.
I20260923 15:05:37.209189 31190 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b7576b0ea7b44739b14360a89cff626e: Neither blocks nor log segments found. Creating new log.
I20260923 15:05:37.209556 31190 log.cc:824] T 00000000000000000000000000000000 P b7576b0ea7b44739b14360a89cff626e: Log is configured to *not* fsync() on all Append() calls
I20260923 15:05:37.210412 31190 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b7576b0ea7b44739b14360a89cff626e: No bootstrap required, opened a new log
I20260923 15:05:37.211499 31193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:37.204958 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42604 (local address 127.13.75.190:43935)
0923 15:05:37.205195 (+ 237us) server_negotiation.cc:206] Beginning negotiation
0923 15:05:37.205205 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0923 15:05:37.205247 (+ 42us) server_negotiation.cc:407] Connection header received
0923 15:05:37.205722 (+ 475us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:05:37.205732 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:05:37.205828 (+ 96us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:05:37.206008 (+ 180us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:05:37.206287 (+ 279us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:05:37.207823 (+ 1536us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:05:37.208750 (+ 927us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:05:37.209483 (+ 733us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:05:37.209644 (+ 161us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:05:37.210626 (+ 982us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:05:37.210630 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:05:37.210631 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 15:05:37.210851 (+ 220us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:05:37.210922 (+ 71us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 15:05:37.210938 (+ 16us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:05:37.211012 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0923 15:05:37.211184 (+ 172us) server_negotiation.cc:299] Negotiation successful
0923 15:05:37.211274 (+ 90us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":147,"thread_start_us":82,"threads_started":1}
I20260923 15:05:37.212878 31190 raft_consensus.cc:348] T 00000000000000000000000000000000 P b7576b0ea7b44739b14360a89cff626e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7576b0ea7b44739b14360a89cff626e" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 43935 } }
I20260923 15:05:37.213027 31190 raft_consensus.cc:374] T 00000000000000000000000000000000 P b7576b0ea7b44739b14360a89cff626e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:05:37.213048 31190 raft_consensus.cc:729] T 00000000000000000000000000000000 P b7576b0ea7b44739b14360a89cff626e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b7576b0ea7b44739b14360a89cff626e, State: Initialized, Role: FOLLOWER
I20260923 15:05:37.213133 31190 consensus_queue.cc:260] T 00000000000000000000000000000000 P b7576b0ea7b44739b14360a89cff626e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7576b0ea7b44739b14360a89cff626e" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 43935 } }
I20260923 15:05:37.213202 31190 raft_consensus.cc:388] T 00000000000000000000000000000000 P b7576b0ea7b44739b14360a89cff626e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 15:05:37.213240 31190 raft_consensus.cc:482] T 00000000000000000000000000000000 P b7576b0ea7b44739b14360a89cff626e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 15:05:37.213279 31190 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b7576b0ea7b44739b14360a89cff626e [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:05:37.213905 31190 raft_consensus.cc:504] T 00000000000000000000000000000000 P b7576b0ea7b44739b14360a89cff626e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7576b0ea7b44739b14360a89cff626e" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 43935 } }
I20260923 15:05:37.214017 31190 leader_election.cc:302] T 00000000000000000000000000000000 P b7576b0ea7b44739b14360a89cff626e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b7576b0ea7b44739b14360a89cff626e; no voters:
I20260923 15:05:37.214192 31190 leader_election.cc:288] T 00000000000000000000000000000000 P b7576b0ea7b44739b14360a89cff626e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 15:05:37.214367 31195 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b7576b0ea7b44739b14360a89cff626e [term 1 FOLLOWER]: Leader election won for term 1
I20260923 15:05:37.214442 31190 sys_catalog.cc:564] T 00000000000000000000000000000000 P b7576b0ea7b44739b14360a89cff626e [sys.catalog]: configured and running, proceeding with master startup.
I20260923 15:05:37.214514 31195 raft_consensus.cc:686] T 00000000000000000000000000000000 P b7576b0ea7b44739b14360a89cff626e [term 1 LEADER]: Becoming Leader. State: Replica: b7576b0ea7b44739b14360a89cff626e, State: Running, Role: LEADER
I20260923 15:05:37.214617 31195 consensus_queue.cc:237] T 00000000000000000000000000000000 P b7576b0ea7b44739b14360a89cff626e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7576b0ea7b44739b14360a89cff626e" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 43935 } }
I20260923 15:05:37.215042 31196 sys_catalog.cc:455] T 00000000000000000000000000000000 P b7576b0ea7b44739b14360a89cff626e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b7576b0ea7b44739b14360a89cff626e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7576b0ea7b44739b14360a89cff626e" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 43935 } } }
I20260923 15:05:37.215142 31196 sys_catalog.cc:458] T 00000000000000000000000000000000 P b7576b0ea7b44739b14360a89cff626e [sys.catalog]: This master's current role is: LEADER
I20260923 15:05:37.215363 31197 sys_catalog.cc:455] T 00000000000000000000000000000000 P b7576b0ea7b44739b14360a89cff626e [sys.catalog]: SysCatalogTable state changed. Reason: New leader b7576b0ea7b44739b14360a89cff626e. Latest consensus state: current_term: 1 leader_uuid: "b7576b0ea7b44739b14360a89cff626e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b7576b0ea7b44739b14360a89cff626e" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 43935 } } }
I20260923 15:05:37.215368 31204 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 15:05:37.215963 31204 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 15:05:37.215466 31197 sys_catalog.cc:458] T 00000000000000000000000000000000 P b7576b0ea7b44739b14360a89cff626e [sys.catalog]: This master's current role is: LEADER
I20260923 15:05:37.217587 31204 catalog_manager.cc:1347] Generated new cluster ID: d83c25734154434dbc29115a564befa0
I20260923 15:05:37.217645 31204 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 15:05:37.232386 31204 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 15:05:37.233188 31204 catalog_manager.cc:1504] Loading token signing keys...
I20260923 15:05:37.238142 31204 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b7576b0ea7b44739b14360a89cff626e: Generated new TSK 0
I20260923 15:05:37.238375 31204 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 15:05:37.243997 13614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKC4soy/build/release/bin/kudu
/tmp/dist-test-taskKC4soy/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790175691491353-13614-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790175691491353-13614-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790175691491353-13614-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790175691491353-13614-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.75.129:0
--local_ip_for_outbound_sockets=127.13.75.129
--webserver_interface=127.13.75.129
--webserver_port=0
--tserver_master_addrs=127.13.75.190:43935
--builtin_ntp_servers=127.13.75.148:34193
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260923 15:05:37.326714 31214 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:05:37.327028 31214 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:05:37.327072 31214 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:05:37.328316 31214 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:05:37.328392 31214 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:05:37.328437 31214 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.75.129
I20260923 15:05:37.329890 31214 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.75.148:34193
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790175691491353-13614-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790175691491353-13614-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.75.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790175691491353-13614-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.75.129
--webserver_port=0
--tserver_master_addrs=127.13.75.190:43935
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31214
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.75.129
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790175691491353-13614-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:05:37.330188 31214 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:05:37.330533 31214 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:05:37.335944 31219 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:05:37.336048 31214 server_base.cc:1034] running on GCE node
W20260923 15:05:37.336158 31222 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:05:37.335958 31220 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:05:37.336483 31214 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:05:37.336761 31214 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:05:37.338055 31214 hybrid_clock.cc:648] HybridClock initialized: now 1790175937338024 us; error 45 us; skew 500 ppm
I20260923 15:05:37.339562 31214 webserver.cc:460] Webserver started at http://127.13.75.129:38279/ using document root <none> and password file <none>
I20260923 15:05:37.339864 31214 fs_manager.cc:359] Metadata directory not provided
I20260923 15:05:37.339941 31214 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:05:37.340106 31214 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:05:37.341310 31214 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790175691491353-13614-0/minicluster-data/ts-0/data/instance:
uuid: "c9aa83ce8d4a469093a495efae0409da"
format_stamp: "Formatted at 2026-09-23 15:05:37 on dist-test-slave-cnb0"
server_key: "1b4867a77394cd620fbe9a759bcad491"
server_key_iv: "dc238ea230b8ca029c4d2274679cc68a"
server_key_version: "encryptionkey@0"
I20260923 15:05:37.341684 31214 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790175691491353-13614-0/minicluster-data/ts-0/wal/instance:
uuid: "c9aa83ce8d4a469093a495efae0409da"
format_stamp: "Formatted at 2026-09-23 15:05:37 on dist-test-slave-cnb0"
server_key: "1b4867a77394cd620fbe9a759bcad491"
server_key_iv: "dc238ea230b8ca029c4d2274679cc68a"
server_key_version: "encryptionkey@0"
I20260923 15:05:37.343196 31214 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260923 15:05:37.344098 31228 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:05:37.344316 31214 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:05:37.344427 31214 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790175691491353-13614-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790175691491353-13614-0/minicluster-data/ts-0/wal
uuid: "c9aa83ce8d4a469093a495efae0409da"
format_stamp: "Formatted at 2026-09-23 15:05:37 on dist-test-slave-cnb0"
server_key: "1b4867a77394cd620fbe9a759bcad491"
server_key_iv: "dc238ea230b8ca029c4d2274679cc68a"
server_key_version: "encryptionkey@0"
I20260923 15:05:37.344509 31214 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790175691491353-13614-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790175691491353-13614-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790175691491353-13614-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:05:37.384570 31214 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:05:37.384958 31214 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:05:37.385128 31214 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:05:37.385394 31214 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 15:05:37.385746 31214 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 15:05:37.385802 31214 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 15:05:37.385844 31214 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 15:05:37.385876 31214 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 15:05:37.391953 31214 rpc_server.cc:307] RPC server started. Bound to: 127.13.75.129:38121
I20260923 15:05:37.392000 31341 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.75.129:38121 every 8 connection(s)
I20260923 15:05:37.392347 31214 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790175691491353-13614-0/minicluster-data/ts-0/data/info.pb
I20260923 15:05:37.397778 31193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:37.393133 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.129:56249 (local address 127.13.75.190:43935)
0923 15:05:37.393226 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0923 15:05:37.393229 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:05:37.393456 (+ 227us) server_negotiation.cc:407] Connection header received
0923 15:05:37.393988 (+ 532us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:05:37.393989 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:05:37.393999 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:05:37.394056 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:05:37.394783 (+ 727us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:05:37.395344 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:05:37.396338 (+ 994us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:05:37.396521 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:05:37.396575 (+ 54us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:05:37.397468 (+ 893us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:05:37.397469 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:05:37.397469 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 15:05:37.397474 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:05:37.397488 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 15:05:37.397492 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:05:37.397539 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0923 15:05:37.397660 (+ 121us) server_negotiation.cc:299] Negotiation successful
0923 15:05:37.397696 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260923 15:05:37.398025 31344 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:37.393084 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:43935 (local address 127.13.75.129:56249)
0923 15:05:37.393316 (+ 232us) negotiation.cc:107] Waiting for socket to connect
0923 15:05:37.393362 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0923 15:05:37.393899 (+ 537us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:05:37.394130 (+ 231us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:05:37.394133 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:05:37.394149 (+ 16us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:05:37.394659 (+ 510us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:05:37.394664 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:05:37.395432 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:05:37.395433 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:05:37.396261 (+ 828us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:05:37.396264 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:05:37.396598 (+ 334us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:05:37.396599 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:05:37.396727 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:05:37.397315 (+ 588us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 15:05:37.397316 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:05:37.397347 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 15:05:37.397372 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:05:37.397574 (+ 202us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:05:37.397581 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:05:37.397584 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 15:05:37.397658 (+ 74us) client_negotiation.cc:240] Negotiation successful
0923 15:05:37.397752 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":157,"thread_start_us":103,"threads_started":1}
I20260923 15:05:37.398785 31342 heartbeater.cc:347] Connected to a master server at 127.13.75.190:43935
I20260923 15:05:37.398941 31342 heartbeater.cc:464] Registering TS with master...
I20260923 15:05:37.399412 31342 heartbeater.cc:511] Master 127.13.75.190:43935 requested a full tablet report, sending...
I20260923 15:05:37.399914 13614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKC4soy/build/release/bin/kudu as pid 31214
I20260923 15:05:37.399999 13614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790175691491353-13614-0/minicluster-data/ts-0/wal/instance
I20260923 15:05:37.400169 13614 external_mini_cluster.cc:1428] Setting key 31624d8d59bee7482594b05fb1e0febb
I20260923 15:05:37.400295 31154 ts_manager.cc:194] Registered new tserver with Master: c9aa83ce8d4a469093a495efae0409da (127.13.75.129:38121)
I20260923 15:05:37.401324 31154 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.13.75.129:56249
I20260923 15:05:37.401401 13614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKC4soy/build/release/bin/kudu
/tmp/dist-test-taskKC4soy/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790175691491353-13614-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790175691491353-13614-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790175691491353-13614-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790175691491353-13614-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.75.130:0
--local_ip_for_outbound_sockets=127.13.75.130
--webserver_interface=127.13.75.130
--webserver_port=0
--tserver_master_addrs=127.13.75.190:43935
--builtin_ntp_servers=127.13.75.148:34193
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260923 15:05:37.484107 31345 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:05:37.484328 31345 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:05:37.484351 31345 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:05:37.485484 31345 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:05:37.485525 31345 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:05:37.485551 31345 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.75.130
I20260923 15:05:37.486871 31345 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.75.148:34193
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790175691491353-13614-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790175691491353-13614-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.75.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790175691491353-13614-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.75.130
--webserver_port=0
--tserver_master_addrs=127.13.75.190:43935
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31345
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.75.130
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790175691491353-13614-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:05:37.487097 31345 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:05:37.487356 31345 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:05:37.492386 31350 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:05:37.492533 31345 server_base.cc:1034] running on GCE node
W20260923 15:05:37.492379 31351 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:05:37.492413 31353 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:05:37.492955 31345 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:05:37.493218 31345 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:05:37.494374 31345 hybrid_clock.cc:648] HybridClock initialized: now 1790175937494361 us; error 31 us; skew 500 ppm
I20260923 15:05:37.495847 31345 webserver.cc:460] Webserver started at http://127.13.75.130:37955/ using document root <none> and password file <none>
I20260923 15:05:37.496126 31345 fs_manager.cc:359] Metadata directory not provided
I20260923 15:05:37.496201 31345 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:05:37.496389 31345 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:05:37.497479 31345 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790175691491353-13614-0/minicluster-data/ts-1/data/instance:
uuid: "e14c426896764ab581b7d98706ccf697"
format_stamp: "Formatted at 2026-09-23 15:05:37 on dist-test-slave-cnb0"
server_key: "59febb9e689206e74d8c32744f808011"
server_key_iv: "ec0c470f5f0ead90780e43b77afdaa3e"
server_key_version: "encryptionkey@0"
I20260923 15:05:37.497848 31345 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790175691491353-13614-0/minicluster-data/ts-1/wal/instance:
uuid: "e14c426896764ab581b7d98706ccf697"
format_stamp: "Formatted at 2026-09-23 15:05:37 on dist-test-slave-cnb0"
server_key: "59febb9e689206e74d8c32744f808011"
server_key_iv: "ec0c470f5f0ead90780e43b77afdaa3e"
server_key_version: "encryptionkey@0"
I20260923 15:05:37.499439 31345 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 15:05:37.500289 31359 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:05:37.500494 31345 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:05:37.500591 31345 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790175691491353-13614-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790175691491353-13614-0/minicluster-data/ts-1/wal
uuid: "e14c426896764ab581b7d98706ccf697"
format_stamp: "Formatted at 2026-09-23 15:05:37 on dist-test-slave-cnb0"
server_key: "59febb9e689206e74d8c32744f808011"
server_key_iv: "ec0c470f5f0ead90780e43b77afdaa3e"
server_key_version: "encryptionkey@0"
I20260923 15:05:37.500667 31345 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790175691491353-13614-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790175691491353-13614-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790175691491353-13614-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:05:37.521535 31345 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:05:37.521899 31345 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:05:37.522071 31345 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:05:37.522334 31345 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 15:05:37.522683 31345 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 15:05:37.522719 31345 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 15:05:37.522774 31345 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 15:05:37.522812 31345 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 15:05:37.528954 31345 rpc_server.cc:307] RPC server started. Bound to: 127.13.75.130:40211
I20260923 15:05:37.528997 31472 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.75.130:40211 every 8 connection(s)
I20260923 15:05:37.529357 31345 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790175691491353-13614-0/minicluster-data/ts-1/data/info.pb
I20260923 15:05:37.535099 31193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:37.530129 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.130:52469 (local address 127.13.75.190:43935)
0923 15:05:37.530218 (+ 89us) server_negotiation.cc:206] Beginning negotiation
0923 15:05:37.530220 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:05:37.530489 (+ 269us) server_negotiation.cc:407] Connection header received
0923 15:05:37.531056 (+ 567us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:05:37.531057 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:05:37.531068 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:05:37.531138 (+ 70us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:05:37.531883 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:05:37.532471 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:05:37.533574 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:05:37.533781 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:05:37.533845 (+ 64us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:05:37.534736 (+ 891us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:05:37.534738 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:05:37.534738 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 15:05:37.534743 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:05:37.534757 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 15:05:37.534761 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:05:37.534822 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0923 15:05:37.535006 (+ 184us) server_negotiation.cc:299] Negotiation successful
0923 15:05:37.535030 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":60}
I20260923 15:05:37.535393 31475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:37.530108 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:43935 (local address 127.13.75.130:52469)
0923 15:05:37.530354 (+ 246us) negotiation.cc:107] Waiting for socket to connect
0923 15:05:37.530396 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0923 15:05:37.530956 (+ 560us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:05:37.531202 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:05:37.531205 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:05:37.531215 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:05:37.531749 (+ 534us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:05:37.531754 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:05:37.532606 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:05:37.532607 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:05:37.533479 (+ 872us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:05:37.533482 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:05:37.533885 (+ 403us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:05:37.533886 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:05:37.534009 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:05:37.534542 (+ 533us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 15:05:37.534542 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:05:37.534600 (+ 58us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 15:05:37.534636 (+ 36us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:05:37.534850 (+ 214us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:05:37.534857 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:05:37.534860 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 15:05:37.535005 (+ 145us) client_negotiation.cc:240] Negotiation successful
0923 15:05:37.535114 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":132,"threads_started":1}
I20260923 15:05:37.536228 31473 heartbeater.cc:347] Connected to a master server at 127.13.75.190:43935
I20260923 15:05:37.536360 31473 heartbeater.cc:464] Registering TS with master...
I20260923 15:05:37.536563 31473 heartbeater.cc:511] Master 127.13.75.190:43935 requested a full tablet report, sending...
I20260923 15:05:37.537001 13614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKC4soy/build/release/bin/kudu as pid 31345
I20260923 15:05:37.537091 13614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790175691491353-13614-0/minicluster-data/ts-1/wal/instance
I20260923 15:05:37.537084 31154 ts_manager.cc:194] Registered new tserver with Master: e14c426896764ab581b7d98706ccf697 (127.13.75.130:40211)
I20260923 15:05:37.537317 13614 external_mini_cluster.cc:1428] Setting key 73d491b442b82ccd67a6185e65aaaa3b
I20260923 15:05:37.537590 31154 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.13.75.130:52469
I20260923 15:05:37.538376 13614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKC4soy/build/release/bin/kudu
/tmp/dist-test-taskKC4soy/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790175691491353-13614-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790175691491353-13614-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790175691491353-13614-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790175691491353-13614-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.75.131:0
--local_ip_for_outbound_sockets=127.13.75.131
--webserver_interface=127.13.75.131
--webserver_port=0
--tserver_master_addrs=127.13.75.190:43935
--builtin_ntp_servers=127.13.75.148:34193
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260923 15:05:37.624915 31476 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:05:37.625126 31476 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:05:37.625149 31476 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:05:37.626271 31476 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:05:37.626308 31476 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:05:37.626334 31476 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.75.131
I20260923 15:05:37.627746 31476 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.75.148:34193
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790175691491353-13614-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790175691491353-13614-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.75.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790175691491353-13614-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.75.131
--webserver_port=0
--tserver_master_addrs=127.13.75.190:43935
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31476
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.75.131
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790175691491353-13614-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:05:37.627952 31476 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:05:37.628206 31476 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:05:37.633589 31481 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:05:37.633682 31476 server_base.cc:1034] running on GCE node
W20260923 15:05:37.633589 31484 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:05:37.633597 31482 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:05:37.634028 31476 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:05:37.634338 31476 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:05:37.635529 31476 hybrid_clock.cc:648] HybridClock initialized: now 1790175937635516 us; error 39 us; skew 500 ppm
I20260923 15:05:37.636819 31476 webserver.cc:460] Webserver started at http://127.13.75.131:33355/ using document root <none> and password file <none>
I20260923 15:05:37.637070 31476 fs_manager.cc:359] Metadata directory not provided
I20260923 15:05:37.637145 31476 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:05:37.637341 31476 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:05:37.638449 31476 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790175691491353-13614-0/minicluster-data/ts-2/data/instance:
uuid: "2344cde268a84619b959687f00355f48"
format_stamp: "Formatted at 2026-09-23 15:05:37 on dist-test-slave-cnb0"
server_key: "9eda23e175b7fd0826bac81b3e1bf64a"
server_key_iv: "076610c39c3615672dd4d2a24a085132"
server_key_version: "encryptionkey@0"
I20260923 15:05:37.638820 31476 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790175691491353-13614-0/minicluster-data/ts-2/wal/instance:
uuid: "2344cde268a84619b959687f00355f48"
format_stamp: "Formatted at 2026-09-23 15:05:37 on dist-test-slave-cnb0"
server_key: "9eda23e175b7fd0826bac81b3e1bf64a"
server_key_iv: "076610c39c3615672dd4d2a24a085132"
server_key_version: "encryptionkey@0"
I20260923 15:05:37.640288 31476 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260923 15:05:37.641125 31490 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:05:37.641366 31476 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260923 15:05:37.641482 31476 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790175691491353-13614-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790175691491353-13614-0/minicluster-data/ts-2/wal
uuid: "2344cde268a84619b959687f00355f48"
format_stamp: "Formatted at 2026-09-23 15:05:37 on dist-test-slave-cnb0"
server_key: "9eda23e175b7fd0826bac81b3e1bf64a"
server_key_iv: "076610c39c3615672dd4d2a24a085132"
server_key_version: "encryptionkey@0"
I20260923 15:05:37.641561 31476 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790175691491353-13614-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790175691491353-13614-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790175691491353-13614-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:05:37.652650 31476 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:05:37.652990 31476 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:05:37.653148 31476 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:05:37.653388 31476 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 15:05:37.653757 31476 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 15:05:37.653815 31476 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 15:05:37.653859 31476 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 15:05:37.653887 31476 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 15:05:37.659907 31476 rpc_server.cc:307] RPC server started. Bound to: 127.13.75.131:36715
I20260923 15:05:37.659983 31603 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.75.131:36715 every 8 connection(s)
I20260923 15:05:37.660399 31476 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790175691491353-13614-0/minicluster-data/ts-2/data/info.pb
I20260923 15:05:37.663856 13614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKC4soy/build/release/bin/kudu as pid 31476
I20260923 15:05:37.663966 13614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790175691491353-13614-0/minicluster-data/ts-2/wal/instance
I20260923 15:05:37.664166 13614 external_mini_cluster.cc:1428] Setting key b4f009cb5f9dd7220c90e2311431dc60
I20260923 15:05:37.668195 31193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:37.661330 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.131:40535 (local address 127.13.75.190:43935)
0923 15:05:37.661406 (+ 76us) server_negotiation.cc:206] Beginning negotiation
0923 15:05:37.661408 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:05:37.661780 (+ 372us) server_negotiation.cc:407] Connection header received
0923 15:05:37.662820 (+ 1040us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:05:37.662821 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:05:37.662836 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:05:37.662962 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:05:37.663999 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:05:37.664762 (+ 763us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:05:37.666324 (+ 1562us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:05:37.666631 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:05:37.666711 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:05:37.667799 (+ 1088us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:05:37.667801 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:05:37.667801 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 15:05:37.667808 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:05:37.667827 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 15:05:37.667832 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:05:37.667892 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0923 15:05:37.668032 (+ 140us) server_negotiation.cc:299] Negotiation successful
0923 15:05:37.668092 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260923 15:05:37.668430 31606 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:37.661352 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:43935 (local address 127.13.75.131:40535)
0923 15:05:37.661627 (+ 275us) negotiation.cc:107] Waiting for socket to connect
0923 15:05:37.661673 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0923 15:05:37.662605 (+ 932us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:05:37.663063 (+ 458us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:05:37.663068 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:05:37.663079 (+ 11us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:05:37.663872 (+ 793us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:05:37.663879 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:05:37.664894 (+ 1015us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:05:37.664895 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:05:37.666213 (+ 1318us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:05:37.666216 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:05:37.666748 (+ 532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:05:37.666750 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:05:37.666952 (+ 202us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:05:37.667602 (+ 650us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 15:05:37.667604 (+ 2us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:05:37.667651 (+ 47us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 15:05:37.667682 (+ 31us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:05:37.667893 (+ 211us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:05:37.667899 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:05:37.667901 (+ 2us) client_negotiation.cc:769] Sending connection context
0923 15:05:37.668029 (+ 128us) client_negotiation.cc:240] Negotiation successful
0923 15:05:37.668145 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":184,"thread_start_us":106,"threads_started":1}
I20260923 15:05:37.668954 31604 heartbeater.cc:347] Connected to a master server at 127.13.75.190:43935
I20260923 15:05:37.669052 31604 heartbeater.cc:464] Registering TS with master...
I20260923 15:05:37.669287 31604 heartbeater.cc:511] Master 127.13.75.190:43935 requested a full tablet report, sending...
I20260923 15:05:37.669709 31153 ts_manager.cc:194] Registered new tserver with Master: 2344cde268a84619b959687f00355f48 (127.13.75.131:36715)
I20260923 15:05:37.670156 31153 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.13.75.131:40535
I20260923 15:05:37.676236 13614 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260923 15:05:37.680790 31193 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:37.677732 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42614 (local address 127.13.75.190:43935)
0923 15:05:37.677815 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0923 15:05:37.677818 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:05:37.677911 (+ 93us) server_negotiation.cc:407] Connection header received
0923 15:05:37.677998 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:05:37.677999 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:05:37.678011 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:05:37.678085 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:05:37.678251 (+ 166us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:05:37.678737 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:05:37.679526 (+ 789us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:05:37.679717 (+ 191us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:05:37.679787 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:05:37.680511 (+ 724us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:05:37.680512 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:05:37.680513 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 15:05:37.680517 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:05:37.680532 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 15:05:37.680536 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:05:37.680609 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0923 15:05:37.680669 (+ 60us) server_negotiation.cc:299] Negotiation successful
0923 15:05:37.680695 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260923 15:05:37.682601 31153 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:42614:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 15:05:37.684171 31153 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 15:05:37.690536 31616 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:37.686574 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.130:40211 (local address 127.0.0.1:59274)
0923 15:05:37.686760 (+ 186us) negotiation.cc:107] Waiting for socket to connect
0923 15:05:37.686769 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 15:05:37.686812 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:05:37.687193 (+ 381us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:05:37.687194 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:05:37.687196 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:05:37.687253 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:05:37.687255 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:05:37.688160 (+ 905us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:05:37.688161 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:05:37.689336 (+ 1175us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:05:37.689338 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:05:37.690255 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:05:37.690256 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:05:37.690331 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:05:37.690336 (+ 5us) client_negotiation.cc:769] Sending connection context
0923 15:05:37.690371 (+ 35us) client_negotiation.cc:240] Negotiation successful
0923 15:05:37.690437 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":173,"mutex_wait_us":1,"thread_start_us":85,"threads_started":1}
I20260923 15:05:37.690532 31614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:37.686486 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:47042 (local address 127.13.75.129:38121)
0923 15:05:37.686882 (+ 396us) server_negotiation.cc:206] Beginning negotiation
0923 15:05:37.686887 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 15:05:37.686930 (+ 43us) server_negotiation.cc:407] Connection header received
0923 15:05:37.686974 (+ 44us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:05:37.686977 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:05:37.686993 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:05:37.687074 (+ 81us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:05:37.687305 (+ 231us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:05:37.688451 (+ 1146us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:05:37.689633 (+ 1182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:05:37.690122 (+ 489us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:05:37.690218 (+ 96us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:05:37.690318 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0923 15:05:37.690410 (+ 92us) server_negotiation.cc:299] Negotiation successful
0923 15:05:37.690446 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":356,"thread_start_us":99,"threads_started":1}
I20260923 15:05:37.690737 31617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:37.686822 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59274 (local address 127.13.75.130:40211)
0923 15:05:37.687014 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0923 15:05:37.687018 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 15:05:37.687029 (+ 11us) server_negotiation.cc:407] Connection header received
0923 15:05:37.687071 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:05:37.687074 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:05:37.687089 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:05:37.687341 (+ 252us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:05:37.687403 (+ 62us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:05:37.688074 (+ 671us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:05:37.689424 (+ 1350us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:05:37.690139 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:05:37.690489 (+ 350us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:05:37.690575 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0923 15:05:37.690599 (+ 24us) server_negotiation.cc:299] Negotiation successful
0923 15:05:37.690644 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":110,"threads_started":1}
I20260923 15:05:37.690760 31613 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:37.686376 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.129:38121 (local address 127.0.0.1:47042)
0923 15:05:37.686496 (+ 120us) negotiation.cc:107] Waiting for socket to connect
0923 15:05:37.686537 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0923 15:05:37.686598 (+ 61us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:05:37.687158 (+ 560us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:05:37.687162 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:05:37.687165 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:05:37.687234 (+ 69us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:05:37.687236 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:05:37.688542 (+ 1306us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:05:37.688543 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:05:37.689557 (+ 1014us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:05:37.689558 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:05:37.690221 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:05:37.690222 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:05:37.690308 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:05:37.690322 (+ 14us) client_negotiation.cc:769] Sending connection context
0923 15:05:37.690391 (+ 69us) client_negotiation.cc:240] Negotiation successful
0923 15:05:37.690436 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":95,"mutex_wait_us":33,"thread_start_us":72,"threads_started":1}
I20260923 15:05:37.690863 31615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:37.686567 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38364 (local address 127.13.75.131:36715)
0923 15:05:37.686704 (+ 137us) server_negotiation.cc:206] Beginning negotiation
0923 15:05:37.686710 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 15:05:37.687038 (+ 328us) server_negotiation.cc:407] Connection header received
0923 15:05:37.687081 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:05:37.687084 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:05:37.687097 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:05:37.687168 (+ 71us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:05:37.687364 (+ 196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:05:37.688451 (+ 1087us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:05:37.689691 (+ 1240us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:05:37.690382 (+ 691us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:05:37.690421 (+ 39us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:05:37.690499 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0923 15:05:37.690701 (+ 202us) server_negotiation.cc:299] Negotiation successful
0923 15:05:37.690753 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":100,"thread_start_us":69,"threads_started":1}
I20260923 15:05:37.691105 31612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:37.686378 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.131:36715 (local address 127.0.0.1:38364)
0923 15:05:37.686713 (+ 335us) negotiation.cc:107] Waiting for socket to connect
0923 15:05:37.686723 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 15:05:37.686804 (+ 81us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:05:37.687236 (+ 432us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:05:37.687237 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:05:37.687239 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:05:37.687266 (+ 27us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:05:37.687267 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:05:37.688553 (+ 1286us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:05:37.688554 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:05:37.689590 (+ 1036us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:05:37.689593 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:05:37.690558 (+ 965us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:05:37.690558 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:05:37.690615 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:05:37.690618 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 15:05:37.690985 (+ 367us) client_negotiation.cc:240] Negotiation successful
0923 15:05:37.691030 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":322,"thread_start_us":55,"threads_started":1}
I20260923 15:05:37.691527 31407 tablet_service.cc:1467] Processing CreateTablet for tablet 377f02b4e94349a1bf182f8b5da15134 (DEFAULT_TABLE table=test-table [id=8930bd872e464103ac810a4d1e3a7bdf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 15:05:37.691913 31407 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 377f02b4e94349a1bf182f8b5da15134. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:05:37.691815 31538 tablet_service.cc:1467] Processing CreateTablet for tablet 377f02b4e94349a1bf182f8b5da15134 (DEFAULT_TABLE table=test-table [id=8930bd872e464103ac810a4d1e3a7bdf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 15:05:37.692018 31276 tablet_service.cc:1467] Processing CreateTablet for tablet 377f02b4e94349a1bf182f8b5da15134 (DEFAULT_TABLE table=test-table [id=8930bd872e464103ac810a4d1e3a7bdf]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 15:05:37.692130 31538 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 377f02b4e94349a1bf182f8b5da15134. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:05:37.692315 31276 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 377f02b4e94349a1bf182f8b5da15134. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:05:37.694690 31618 tablet_bootstrap.cc:492] T 377f02b4e94349a1bf182f8b5da15134 P e14c426896764ab581b7d98706ccf697: Bootstrap starting.
I20260923 15:05:37.695387 31618 tablet_bootstrap.cc:654] T 377f02b4e94349a1bf182f8b5da15134 P e14c426896764ab581b7d98706ccf697: Neither blocks nor log segments found. Creating new log.
I20260923 15:05:37.695523 31619 tablet_bootstrap.cc:492] T 377f02b4e94349a1bf182f8b5da15134 P 2344cde268a84619b959687f00355f48: Bootstrap starting.
I20260923 15:05:37.695847 31618 log.cc:824] T 377f02b4e94349a1bf182f8b5da15134 P e14c426896764ab581b7d98706ccf697: Log is configured to *not* fsync() on all Append() calls
I20260923 15:05:37.696184 31619 tablet_bootstrap.cc:654] T 377f02b4e94349a1bf182f8b5da15134 P 2344cde268a84619b959687f00355f48: Neither blocks nor log segments found. Creating new log.
I20260923 15:05:37.696203 31620 tablet_bootstrap.cc:492] T 377f02b4e94349a1bf182f8b5da15134 P c9aa83ce8d4a469093a495efae0409da: Bootstrap starting.
I20260923 15:05:37.696528 31619 log.cc:824] T 377f02b4e94349a1bf182f8b5da15134 P 2344cde268a84619b959687f00355f48: Log is configured to *not* fsync() on all Append() calls
I20260923 15:05:37.696664 31618 tablet_bootstrap.cc:492] T 377f02b4e94349a1bf182f8b5da15134 P e14c426896764ab581b7d98706ccf697: No bootstrap required, opened a new log
I20260923 15:05:37.696753 31618 ts_tablet_manager.cc:1397] T 377f02b4e94349a1bf182f8b5da15134 P e14c426896764ab581b7d98706ccf697: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 15:05:37.696862 31620 tablet_bootstrap.cc:654] T 377f02b4e94349a1bf182f8b5da15134 P c9aa83ce8d4a469093a495efae0409da: Neither blocks nor log segments found. Creating new log.
I20260923 15:05:37.697149 31620 log.cc:824] T 377f02b4e94349a1bf182f8b5da15134 P c9aa83ce8d4a469093a495efae0409da: Log is configured to *not* fsync() on all Append() calls
I20260923 15:05:37.697865 31620 tablet_bootstrap.cc:492] T 377f02b4e94349a1bf182f8b5da15134 P c9aa83ce8d4a469093a495efae0409da: No bootstrap required, opened a new log
I20260923 15:05:37.697919 31619 tablet_bootstrap.cc:492] T 377f02b4e94349a1bf182f8b5da15134 P 2344cde268a84619b959687f00355f48: No bootstrap required, opened a new log
I20260923 15:05:37.697934 31620 ts_tablet_manager.cc:1397] T 377f02b4e94349a1bf182f8b5da15134 P c9aa83ce8d4a469093a495efae0409da: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 15:05:37.697983 31619 ts_tablet_manager.cc:1397] T 377f02b4e94349a1bf182f8b5da15134 P 2344cde268a84619b959687f00355f48: Time spent bootstrapping tablet: real 0.003s user 0.001s sys 0.000s
I20260923 15:05:37.699011 31618 raft_consensus.cc:348] T 377f02b4e94349a1bf182f8b5da15134 P e14c426896764ab581b7d98706ccf697 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9aa83ce8d4a469093a495efae0409da" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 38121 } } peers { permanent_uuid: "2344cde268a84619b959687f00355f48" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 36715 } } peers { permanent_uuid: "e14c426896764ab581b7d98706ccf697" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 40211 } }
I20260923 15:05:37.699180 31618 raft_consensus.cc:374] T 377f02b4e94349a1bf182f8b5da15134 P e14c426896764ab581b7d98706ccf697 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:05:37.699223 31618 raft_consensus.cc:729] T 377f02b4e94349a1bf182f8b5da15134 P e14c426896764ab581b7d98706ccf697 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e14c426896764ab581b7d98706ccf697, State: Initialized, Role: FOLLOWER
I20260923 15:05:37.699452 31618 consensus_queue.cc:260] T 377f02b4e94349a1bf182f8b5da15134 P e14c426896764ab581b7d98706ccf697 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9aa83ce8d4a469093a495efae0409da" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 38121 } } peers { permanent_uuid: "2344cde268a84619b959687f00355f48" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 36715 } } peers { permanent_uuid: "e14c426896764ab581b7d98706ccf697" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 40211 } }
I20260923 15:05:37.699717 31618 ts_tablet_manager.cc:1428] T 377f02b4e94349a1bf182f8b5da15134 P e14c426896764ab581b7d98706ccf697: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260923 15:05:37.699884 31473 heartbeater.cc:503] Master 127.13.75.190:43935 was elected leader, sending a full tablet report...
I20260923 15:05:37.700531 31620 raft_consensus.cc:348] T 377f02b4e94349a1bf182f8b5da15134 P c9aa83ce8d4a469093a495efae0409da [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9aa83ce8d4a469093a495efae0409da" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 38121 } } peers { permanent_uuid: "2344cde268a84619b959687f00355f48" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 36715 } } peers { permanent_uuid: "e14c426896764ab581b7d98706ccf697" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 40211 } }
I20260923 15:05:37.700531 31619 raft_consensus.cc:348] T 377f02b4e94349a1bf182f8b5da15134 P 2344cde268a84619b959687f00355f48 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9aa83ce8d4a469093a495efae0409da" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 38121 } } peers { permanent_uuid: "2344cde268a84619b959687f00355f48" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 36715 } } peers { permanent_uuid: "e14c426896764ab581b7d98706ccf697" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 40211 } }
I20260923 15:05:37.700693 31619 raft_consensus.cc:374] T 377f02b4e94349a1bf182f8b5da15134 P 2344cde268a84619b959687f00355f48 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:05:37.700693 31620 raft_consensus.cc:374] T 377f02b4e94349a1bf182f8b5da15134 P c9aa83ce8d4a469093a495efae0409da [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:05:37.700742 31620 raft_consensus.cc:729] T 377f02b4e94349a1bf182f8b5da15134 P c9aa83ce8d4a469093a495efae0409da [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c9aa83ce8d4a469093a495efae0409da, State: Initialized, Role: FOLLOWER
I20260923 15:05:37.700742 31619 raft_consensus.cc:729] T 377f02b4e94349a1bf182f8b5da15134 P 2344cde268a84619b959687f00355f48 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 2344cde268a84619b959687f00355f48, State: Initialized, Role: FOLLOWER
I20260923 15:05:37.700866 31620 consensus_queue.cc:260] T 377f02b4e94349a1bf182f8b5da15134 P c9aa83ce8d4a469093a495efae0409da [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9aa83ce8d4a469093a495efae0409da" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 38121 } } peers { permanent_uuid: "2344cde268a84619b959687f00355f48" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 36715 } } peers { permanent_uuid: "e14c426896764ab581b7d98706ccf697" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 40211 } }
I20260923 15:05:37.700866 31619 consensus_queue.cc:260] T 377f02b4e94349a1bf182f8b5da15134 P 2344cde268a84619b959687f00355f48 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9aa83ce8d4a469093a495efae0409da" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 38121 } } peers { permanent_uuid: "2344cde268a84619b959687f00355f48" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 36715 } } peers { permanent_uuid: "e14c426896764ab581b7d98706ccf697" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 40211 } }
I20260923 15:05:37.701124 31620 ts_tablet_manager.cc:1428] T 377f02b4e94349a1bf182f8b5da15134 P c9aa83ce8d4a469093a495efae0409da: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260923 15:05:37.701124 31619 ts_tablet_manager.cc:1428] T 377f02b4e94349a1bf182f8b5da15134 P 2344cde268a84619b959687f00355f48: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260923 15:05:37.701179 31342 heartbeater.cc:503] Master 127.13.75.190:43935 was elected leader, sending a full tablet report...
I20260923 15:05:37.701236 31604 heartbeater.cc:503] Master 127.13.75.190:43935 was elected leader, sending a full tablet report...
W20260923 15:05:37.779912 31474 tablet.cc:2369] T 377f02b4e94349a1bf182f8b5da15134 P e14c426896764ab581b7d98706ccf697: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260923 15:05:37.893100 31343 tablet.cc:2369] T 377f02b4e94349a1bf182f8b5da15134 P c9aa83ce8d4a469093a495efae0409da: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260923 15:05:37.911105 31605 tablet.cc:2369] T 377f02b4e94349a1bf182f8b5da15134 P 2344cde268a84619b959687f00355f48: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 15:05:37.926731 31624 raft_consensus.cc:482] T 377f02b4e94349a1bf182f8b5da15134 P e14c426896764ab581b7d98706ccf697 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 15:05:37.926872 31624 raft_consensus.cc:504] T 377f02b4e94349a1bf182f8b5da15134 P e14c426896764ab581b7d98706ccf697 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9aa83ce8d4a469093a495efae0409da" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 38121 } } peers { permanent_uuid: "2344cde268a84619b959687f00355f48" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 36715 } } peers { permanent_uuid: "e14c426896764ab581b7d98706ccf697" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 40211 } }
I20260923 15:05:37.927500 31624 leader_election.cc:288] T 377f02b4e94349a1bf182f8b5da15134 P e14c426896764ab581b7d98706ccf697 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers c9aa83ce8d4a469093a495efae0409da (127.13.75.129:38121), 2344cde268a84619b959687f00355f48 (127.13.75.131:36715)
I20260923 15:05:37.930549 31614 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:37.927818 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.130:47637 (local address 127.13.75.129:38121)
0923 15:05:37.927863 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0923 15:05:37.927865 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:05:37.928017 (+ 152us) server_negotiation.cc:407] Connection header received
0923 15:05:37.928069 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:05:37.928070 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:05:37.928082 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:05:37.928155 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:05:37.928314 (+ 159us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:05:37.928798 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:05:37.929827 (+ 1029us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:05:37.930190 (+ 363us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:05:37.930276 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:05:37.930284 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0923 15:05:37.930438 (+ 154us) server_negotiation.cc:299] Negotiation successful
0923 15:05:37.930474 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":24}
I20260923 15:05:37.930562 31615 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:37.927830 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.130:36393 (local address 127.13.75.131:36715)
0923 15:05:37.927882 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0923 15:05:37.927884 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:05:37.927897 (+ 13us) server_negotiation.cc:407] Connection header received
0923 15:05:37.928024 (+ 127us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:05:37.928025 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:05:37.928035 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:05:37.928148 (+ 113us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:05:37.928353 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:05:37.928821 (+ 468us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:05:37.929833 (+ 1012us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:05:37.930205 (+ 372us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:05:37.930288 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:05:37.930296 (+ 8us) server_negotiation.cc:1035] Waiting for connection context
0923 15:05:37.930455 (+ 159us) server_negotiation.cc:299] Negotiation successful
0923 15:05:37.930494 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260923 15:05:37.930567 31475 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:37.927737 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.131:36715 (local address 127.13.75.130:36393)
0923 15:05:37.927842 (+ 105us) negotiation.cc:107] Waiting for socket to connect
0923 15:05:37.927857 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0923 15:05:37.927907 (+ 50us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:05:37.928153 (+ 246us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:05:37.928154 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:05:37.928157 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:05:37.928264 (+ 107us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:05:37.928266 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:05:37.928910 (+ 644us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:05:37.928911 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:05:37.929735 (+ 824us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:05:37.929738 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:05:37.930298 (+ 560us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:05:37.930299 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:05:37.930374 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:05:37.930377 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 15:05:37.930437 (+ 60us) client_negotiation.cc:240] Negotiation successful
0923 15:05:37.930484 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":86,"mutex_wait_us":52}
I20260923 15:05:37.930572 31627 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:37.927752 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.129:38121 (local address 127.13.75.130:47637)
0923 15:05:37.927941 (+ 189us) negotiation.cc:107] Waiting for socket to connect
0923 15:05:37.927950 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 15:05:37.928010 (+ 60us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:05:37.928160 (+ 150us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:05:37.928161 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:05:37.928162 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:05:37.928237 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:05:37.928238 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:05:37.928885 (+ 647us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:05:37.928886 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:05:37.929731 (+ 845us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:05:37.929733 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:05:37.930282 (+ 549us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:05:37.930283 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:05:37.930353 (+ 70us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:05:37.930356 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 15:05:37.930430 (+ 74us) client_negotiation.cc:240] Negotiation successful
0923 15:05:37.930472 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"mutex_wait_us":29,"thread_start_us":101,"threads_started":1}
I20260923 15:05:37.930982 31558 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "377f02b4e94349a1bf182f8b5da15134" candidate_uuid: "e14c426896764ab581b7d98706ccf697" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2344cde268a84619b959687f00355f48" is_pre_election: true
I20260923 15:05:37.931176 31558 raft_consensus.cc:2454] T 377f02b4e94349a1bf182f8b5da15134 P 2344cde268a84619b959687f00355f48 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e14c426896764ab581b7d98706ccf697 in term 0.
I20260923 15:05:37.931234 31296 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "377f02b4e94349a1bf182f8b5da15134" candidate_uuid: "e14c426896764ab581b7d98706ccf697" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c9aa83ce8d4a469093a495efae0409da" is_pre_election: true
I20260923 15:05:37.931403 31296 raft_consensus.cc:2454] T 377f02b4e94349a1bf182f8b5da15134 P c9aa83ce8d4a469093a495efae0409da [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate e14c426896764ab581b7d98706ccf697 in term 0.
I20260923 15:05:37.931442 31361 leader_election.cc:302] T 377f02b4e94349a1bf182f8b5da15134 P e14c426896764ab581b7d98706ccf697 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2344cde268a84619b959687f00355f48, e14c426896764ab581b7d98706ccf697; no voters:
I20260923 15:05:37.931563 31624 raft_consensus.cc:2781] T 377f02b4e94349a1bf182f8b5da15134 P e14c426896764ab581b7d98706ccf697 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 15:05:37.931619 31624 raft_consensus.cc:482] T 377f02b4e94349a1bf182f8b5da15134 P e14c426896764ab581b7d98706ccf697 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 15:05:37.931651 31624 raft_consensus.cc:3037] T 377f02b4e94349a1bf182f8b5da15134 P e14c426896764ab581b7d98706ccf697 [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:05:37.932371 31624 raft_consensus.cc:504] T 377f02b4e94349a1bf182f8b5da15134 P e14c426896764ab581b7d98706ccf697 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9aa83ce8d4a469093a495efae0409da" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 38121 } } peers { permanent_uuid: "2344cde268a84619b959687f00355f48" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 36715 } } peers { permanent_uuid: "e14c426896764ab581b7d98706ccf697" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 40211 } }
I20260923 15:05:37.932524 31624 leader_election.cc:288] T 377f02b4e94349a1bf182f8b5da15134 P e14c426896764ab581b7d98706ccf697 [CANDIDATE]: Term 1 election: Requested vote from peers c9aa83ce8d4a469093a495efae0409da (127.13.75.129:38121), 2344cde268a84619b959687f00355f48 (127.13.75.131:36715)
I20260923 15:05:37.932693 31296 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "377f02b4e94349a1bf182f8b5da15134" candidate_uuid: "e14c426896764ab581b7d98706ccf697" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "c9aa83ce8d4a469093a495efae0409da"
I20260923 15:05:37.932749 31558 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "377f02b4e94349a1bf182f8b5da15134" candidate_uuid: "e14c426896764ab581b7d98706ccf697" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "2344cde268a84619b959687f00355f48"
I20260923 15:05:37.932768 31296 raft_consensus.cc:3037] T 377f02b4e94349a1bf182f8b5da15134 P c9aa83ce8d4a469093a495efae0409da [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:05:37.932833 31558 raft_consensus.cc:3037] T 377f02b4e94349a1bf182f8b5da15134 P 2344cde268a84619b959687f00355f48 [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:05:37.933516 31296 raft_consensus.cc:2454] T 377f02b4e94349a1bf182f8b5da15134 P c9aa83ce8d4a469093a495efae0409da [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e14c426896764ab581b7d98706ccf697 in term 1.
I20260923 15:05:37.933547 31558 raft_consensus.cc:2454] T 377f02b4e94349a1bf182f8b5da15134 P 2344cde268a84619b959687f00355f48 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e14c426896764ab581b7d98706ccf697 in term 1.
I20260923 15:05:37.933715 31361 leader_election.cc:302] T 377f02b4e94349a1bf182f8b5da15134 P e14c426896764ab581b7d98706ccf697 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 2344cde268a84619b959687f00355f48, e14c426896764ab581b7d98706ccf697; no voters:
I20260923 15:05:37.933815 31624 raft_consensus.cc:2781] T 377f02b4e94349a1bf182f8b5da15134 P e14c426896764ab581b7d98706ccf697 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 15:05:37.933971 31624 raft_consensus.cc:686] T 377f02b4e94349a1bf182f8b5da15134 P e14c426896764ab581b7d98706ccf697 [term 1 LEADER]: Becoming Leader. State: Replica: e14c426896764ab581b7d98706ccf697, State: Running, Role: LEADER
I20260923 15:05:37.934067 31624 consensus_queue.cc:237] T 377f02b4e94349a1bf182f8b5da15134 P e14c426896764ab581b7d98706ccf697 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9aa83ce8d4a469093a495efae0409da" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 38121 } } peers { permanent_uuid: "2344cde268a84619b959687f00355f48" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 36715 } } peers { permanent_uuid: "e14c426896764ab581b7d98706ccf697" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 40211 } }
I20260923 15:05:37.934935 31153 catalog_manager.cc:5495] T 377f02b4e94349a1bf182f8b5da15134 P e14c426896764ab581b7d98706ccf697 reported cstate change: term changed from 0 to 1, leader changed from <none> to e14c426896764ab581b7d98706ccf697 (127.13.75.130). New cstate: current_term: 1 leader_uuid: "e14c426896764ab581b7d98706ccf697" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c9aa83ce8d4a469093a495efae0409da" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 38121 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "2344cde268a84619b959687f00355f48" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 36715 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e14c426896764ab581b7d98706ccf697" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 40211 } health_report { overall_health: HEALTHY } } }
I20260923 15:05:37.975313 31617 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:37.972733 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:59286 (local address 127.13.75.130:40211)
0923 15:05:37.972808 (+ 75us) server_negotiation.cc:206] Beginning negotiation
0923 15:05:37.972810 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:05:37.972820 (+ 10us) server_negotiation.cc:407] Connection header received
0923 15:05:37.972871 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:05:37.972871 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:05:37.972892 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:05:37.972973 (+ 81us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 15:05:37.973118 (+ 145us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:05:37.973653 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:05:37.974433 (+ 780us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:05:37.974629 (+ 196us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:05:37.974718 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:05:37.974912 (+ 194us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 15:05:37.975024 (+ 112us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 15:05:37.975116 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0923 15:05:37.975170 (+ 54us) server_negotiation.cc:299] Negotiation successful
0923 15:05:37.975194 (+ 24us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":48}
I20260923 15:05:37.977524 31558 raft_consensus.cc:1261] T 377f02b4e94349a1bf182f8b5da15134 P 2344cde268a84619b959687f00355f48 [term 1 FOLLOWER]: Refusing update from remote peer e14c426896764ab581b7d98706ccf697: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 15:05:37.977574 31296 raft_consensus.cc:1261] T 377f02b4e94349a1bf182f8b5da15134 P c9aa83ce8d4a469093a495efae0409da [term 1 FOLLOWER]: Refusing update from remote peer e14c426896764ab581b7d98706ccf697: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 15:05:37.977945 31624 consensus_queue.cc:1035] T 377f02b4e94349a1bf182f8b5da15134 P e14c426896764ab581b7d98706ccf697 [LEADER]: Connected to new peer: Peer: permanent_uuid: "2344cde268a84619b959687f00355f48" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 36715 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 15:05:37.978088 31628 consensus_queue.cc:1035] T 377f02b4e94349a1bf182f8b5da15134 P e14c426896764ab581b7d98706ccf697 [LEADER]: Connected to new peer: Peer: permanent_uuid: "c9aa83ce8d4a469093a495efae0409da" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 38121 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 15:05:37.980756 31631 mvcc.cc:205] Tried to move back new op lower bound from 7332560641952440320 to 7332560641778483200. Current Snapshot: MvccSnapshot[applied={T|T < 7332560641952440320}]
I20260923 15:05:37.983929 31152 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:42614:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 15:05:37.984042 31152 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:42614:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 15:05:37.984918 31152 catalog_manager.cc:5782] T 00000000000000000000000000000000 P b7576b0ea7b44739b14360a89cff626e: Sending DeleteTablet for 3 replicas of tablet 377f02b4e94349a1bf182f8b5da15134
I20260923 15:05:37.985268 31276 tablet_service.cc:1514] Processing DeleteTablet for tablet 377f02b4e94349a1bf182f8b5da15134 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 15:05:37 UTC) from {username='slave'} at 127.0.0.1:47042
I20260923 15:05:37.985424 31538 tablet_service.cc:1514] Processing DeleteTablet for tablet 377f02b4e94349a1bf182f8b5da15134 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 15:05:37 UTC) from {username='slave'} at 127.0.0.1:38364
I20260923 15:05:37.985419 31407 tablet_service.cc:1514] Processing DeleteTablet for tablet 377f02b4e94349a1bf182f8b5da15134 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 15:05:37 UTC) from {username='slave'} at 127.0.0.1:59274
I20260923 15:05:37.985548 31640 tablet_replica.cc:330] T 377f02b4e94349a1bf182f8b5da15134 P c9aa83ce8d4a469093a495efae0409da: stopping tablet replica
I20260923 15:05:37.985656 31640 raft_consensus.cc:2229] T 377f02b4e94349a1bf182f8b5da15134 P c9aa83ce8d4a469093a495efae0409da [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 15:05:37.985687 31642 tablet_replica.cc:330] T 377f02b4e94349a1bf182f8b5da15134 P e14c426896764ab581b7d98706ccf697: stopping tablet replica
I20260923 15:05:37.985785 31640 raft_consensus.cc:2258] T 377f02b4e94349a1bf182f8b5da15134 P c9aa83ce8d4a469093a495efae0409da [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 15:05:37.985787 31642 raft_consensus.cc:2229] T 377f02b4e94349a1bf182f8b5da15134 P e14c426896764ab581b7d98706ccf697 [term 1 LEADER]: Raft consensus shutting down.
I20260923 15:05:37.986017 31642 raft_consensus.cc:2258] T 377f02b4e94349a1bf182f8b5da15134 P e14c426896764ab581b7d98706ccf697 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 15:05:37.986250 31640 ts_tablet_manager.cc:1905] T 377f02b4e94349a1bf182f8b5da15134 P c9aa83ce8d4a469093a495efae0409da: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 15:05:37.986259 13614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKC4soy/build/release/bin/kudu with pid 31214
I20260923 15:05:37.989831 31642 ts_tablet_manager.cc:1905] T 377f02b4e94349a1bf182f8b5da15134 P e14c426896764ab581b7d98706ccf697: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 15:05:37.991042 31641 tablet_replica.cc:330] T 377f02b4e94349a1bf182f8b5da15134 P 2344cde268a84619b959687f00355f48: stopping tablet replica
I20260923 15:05:37.991209 31641 raft_consensus.cc:2229] T 377f02b4e94349a1bf182f8b5da15134 P 2344cde268a84619b959687f00355f48 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 15:05:37.991369 31641 raft_consensus.cc:2258] T 377f02b4e94349a1bf182f8b5da15134 P 2344cde268a84619b959687f00355f48 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 15:05:37.991431 31642 ts_tablet_manager.cc:1918] T 377f02b4e94349a1bf182f8b5da15134 P e14c426896764ab581b7d98706ccf697: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 15:05:37.991508 31642 log.cc:1196] T 377f02b4e94349a1bf182f8b5da15134 P e14c426896764ab581b7d98706ccf697: Deleting WAL directory at /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790175691491353-13614-0/minicluster-data/ts-1/wal/wals/377f02b4e94349a1bf182f8b5da15134
I20260923 15:05:37.991801 31642 ts_tablet_manager.cc:1939] T 377f02b4e94349a1bf182f8b5da15134 P e14c426896764ab581b7d98706ccf697: Deleting consensus metadata
I20260923 15:05:37.991860 31641 ts_tablet_manager.cc:1905] T 377f02b4e94349a1bf182f8b5da15134 P 2344cde268a84619b959687f00355f48: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 15:05:37.992226 31141 catalog_manager.cc:4841] TS e14c426896764ab581b7d98706ccf697 (127.13.75.130:40211): tablet 377f02b4e94349a1bf182f8b5da15134 (table test-table [id=8930bd872e464103ac810a4d1e3a7bdf]) successfully deleted
I20260923 15:05:37.993453 31641 ts_tablet_manager.cc:1918] T 377f02b4e94349a1bf182f8b5da15134 P 2344cde268a84619b959687f00355f48: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 15:05:37.993566 31641 log.cc:1196] T 377f02b4e94349a1bf182f8b5da15134 P 2344cde268a84619b959687f00355f48: Deleting WAL directory at /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_1.1790175691491353-13614-0/minicluster-data/ts-2/wal/wals/377f02b4e94349a1bf182f8b5da15134
I20260923 15:05:37.993829 31641 ts_tablet_manager.cc:1939] T 377f02b4e94349a1bf182f8b5da15134 P 2344cde268a84619b959687f00355f48: Deleting consensus metadata
I20260923 15:05:37.994350 31139 catalog_manager.cc:4841] TS 2344cde268a84619b959687f00355f48 (127.13.75.131:36715): tablet 377f02b4e94349a1bf182f8b5da15134 (table test-table [id=8930bd872e464103ac810a4d1e3a7bdf]) successfully deleted
W20260923 15:05:37.994580 31141 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.75.129:38121 (error 108)
I20260923 15:05:37.994751 13614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKC4soy/build/release/bin/kudu with pid 31345
I20260923 15:05:37.996580 31612 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:37.996416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.129:38121 (local address 127.0.0.1:47056)
0923 15:05:37.996495 (+ 79us) negotiation.cc:107] Waiting for socket to connect
0923 15:05:37.996548 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.129:38121: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260923 15:05:37.996799 31141 catalog_manager.cc:4584] TS c9aa83ce8d4a469093a495efae0409da (127.13.75.129:38121): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 377f02b4e94349a1bf182f8b5da15134: Network error: Client connection negotiation failed: client connection to 127.13.75.129:38121: connect: Connection refused (error 111)
I20260923 15:05:38.000401 13614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKC4soy/build/release/bin/kudu with pid 31476
I20260923 15:05:38.005297 13614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKC4soy/build/release/bin/kudu with pid 31123
2026-09-23T15:05:38Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/1 (954 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/2
Loading random data
Initializing database '/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175691491353-13614-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 15:05:38 dist-test-slave-cnb0 krb5kdc[31646](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 15:05:38 dist-test-slave-cnb0 krb5kdc[31646](info): set up 2 sockets
Sep 23 15:05:38 dist-test-slave-cnb0 krb5kdc[31646](info): commencing operation
krb5kdc: starting...
W20260923 15:05:39.064002 13614 mini_kdc.cc:121] Time spent starting KDC: real 1.038s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175691491353-13614-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175691491353-13614-0/krb5kdc/test-user.keytab.
Sep 23 15:05:39 dist-test-slave-cnb0 krb5kdc[31646](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175939, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T15:05:39Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T15:05:39Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.75.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.75.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.75.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.190.keytab.
Entry for principal kudu/127.13.75.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.190.keytab.
I20260923 15:05:39.190551 13614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKC4soy/build/release/bin/kudu
/tmp/dist-test-taskKC4soy/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175691491353-13614-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175691491353-13614-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175691491353-13614-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175691491353-13614-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.75.190:34895
--webserver_interface=127.13.75.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.190.keytab
--principal=kudu/127.13.75.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.75.148:34427
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.75.190:34895
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175691491353-13614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175691491353-13614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175691491353-13614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:05:39.274420 31659 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:05:39.274659 31659 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:05:39.274685 31659 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:05:39.276304 31659 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 15:05:39.276379 31659 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:05:39.276397 31659 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:05:39.276412 31659 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 15:05:39.276427 31659 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 15:05:39.278188 31659 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.75.148:34427
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175691491353-13614-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175691491353-13614-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.75.190:34895
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.190.keytab
--principal=kudu/127.13.75.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.75.190:34895
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175691491353-13614-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.75.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31659
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175691491353-13614-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:05:39.278470 31659 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:05:39.278692 31659 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260923 15:05:39.284168 31659 server_base.cc:1034] running on GCE node
W20260923 15:05:39.284145 31667 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:05:39.284124 31664 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:05:39.284138 31665 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:05:39.284757 31659 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:05:39.285075 31659 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:05:39.286283 31659 hybrid_clock.cc:648] HybridClock initialized: now 1790175939286250 us; error 49 us; skew 500 ppm
Sep 23 15:05:39 dist-test-slave-cnb0 krb5kdc[31646](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175939, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:05:39.289001 31659 init.cc:377] Logged in from keytab as kudu/127.13.75.190@KRBTEST.COM (short username kudu)
I20260923 15:05:39.289575 31659 webserver.cc:460] Webserver started at http://127.13.75.190:36957/ using document root <none> and password file <none>
I20260923 15:05:39.289813 31659 fs_manager.cc:359] Metadata directory not provided
I20260923 15:05:39.289891 31659 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:05:39.290047 31659 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:05:39.291262 31659 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175691491353-13614-0/minicluster-data/master-0/data/instance:
uuid: "9bcdf9a29ef74caea466640f41ae92a1"
format_stamp: "Formatted at 2026-09-23 15:05:39 on dist-test-slave-cnb0"
server_key: "b56de8e0c5c6f819ce7f80aaccd7c505"
server_key_iv: "aea5fa33f9d083638cd19ed46f97eae8"
server_key_version: "encryptionkey@0"
I20260923 15:05:39.291635 31659 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175691491353-13614-0/minicluster-data/master-0/wal/instance:
uuid: "9bcdf9a29ef74caea466640f41ae92a1"
format_stamp: "Formatted at 2026-09-23 15:05:39 on dist-test-slave-cnb0"
server_key: "b56de8e0c5c6f819ce7f80aaccd7c505"
server_key_iv: "aea5fa33f9d083638cd19ed46f97eae8"
server_key_version: "encryptionkey@0"
I20260923 15:05:39.293044 31659 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 15:05:39.293859 31674 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:05:39.294077 31659 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:05:39.294256 31659 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175691491353-13614-0/minicluster-data/master-0/data,/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175691491353-13614-0/minicluster-data/master-0/wal
uuid: "9bcdf9a29ef74caea466640f41ae92a1"
format_stamp: "Formatted at 2026-09-23 15:05:39 on dist-test-slave-cnb0"
server_key: "b56de8e0c5c6f819ce7f80aaccd7c505"
server_key_iv: "aea5fa33f9d083638cd19ed46f97eae8"
server_key_version: "encryptionkey@0"
I20260923 15:05:39.294341 31659 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175691491353-13614-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175691491353-13614-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175691491353-13614-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:05:39.312635 31659 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:05:39.315296 31659 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:05:39.315486 31659 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:05:39.320065 31659 rpc_server.cc:307] RPC server started. Bound to: 127.13.75.190:34895
I20260923 15:05:39.320112 31726 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.75.190:34895 every 8 connection(s)
I20260923 15:05:39.320516 31659 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175691491353-13614-0/minicluster-data/master-0/data/info.pb
I20260923 15:05:39.321447 31727 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:05:39.324030 31727 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9bcdf9a29ef74caea466640f41ae92a1: Bootstrap starting.
I20260923 15:05:39.324596 31727 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 9bcdf9a29ef74caea466640f41ae92a1: Neither blocks nor log segments found. Creating new log.
I20260923 15:05:39.324862 31727 log.cc:824] T 00000000000000000000000000000000 P 9bcdf9a29ef74caea466640f41ae92a1: Log is configured to *not* fsync() on all Append() calls
I20260923 15:05:39.325588 31727 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 9bcdf9a29ef74caea466640f41ae92a1: No bootstrap required, opened a new log
I20260923 15:05:39.325879 13614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKC4soy/build/release/bin/kudu as pid 31659
I20260923 15:05:39.326010 13614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175691491353-13614-0/minicluster-data/master-0/wal/instance
I20260923 15:05:39.326193 13614 external_mini_cluster.cc:1428] Setting key 9f47c2caefecd233e455aa80e6fdef2f
I20260923 15:05:39.328104 31727 raft_consensus.cc:348] T 00000000000000000000000000000000 P 9bcdf9a29ef74caea466640f41ae92a1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9bcdf9a29ef74caea466640f41ae92a1" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 34895 } }
I20260923 15:05:39.328302 31727 raft_consensus.cc:374] T 00000000000000000000000000000000 P 9bcdf9a29ef74caea466640f41ae92a1 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:05:39.328357 31727 raft_consensus.cc:729] T 00000000000000000000000000000000 P 9bcdf9a29ef74caea466640f41ae92a1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 9bcdf9a29ef74caea466640f41ae92a1, State: Initialized, Role: FOLLOWER
I20260923 15:05:39.328503 31727 consensus_queue.cc:260] T 00000000000000000000000000000000 P 9bcdf9a29ef74caea466640f41ae92a1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9bcdf9a29ef74caea466640f41ae92a1" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 34895 } }
I20260923 15:05:39.328608 31727 raft_consensus.cc:388] T 00000000000000000000000000000000 P 9bcdf9a29ef74caea466640f41ae92a1 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 15:05:39.328685 31727 raft_consensus.cc:482] T 00000000000000000000000000000000 P 9bcdf9a29ef74caea466640f41ae92a1 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 15:05:39.328749 31727 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 9bcdf9a29ef74caea466640f41ae92a1 [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:05:39.329341 31727 raft_consensus.cc:504] T 00000000000000000000000000000000 P 9bcdf9a29ef74caea466640f41ae92a1 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9bcdf9a29ef74caea466640f41ae92a1" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 34895 } }
I20260923 15:05:39.329432 31727 leader_election.cc:302] T 00000000000000000000000000000000 P 9bcdf9a29ef74caea466640f41ae92a1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 9bcdf9a29ef74caea466640f41ae92a1; no voters:
I20260923 15:05:39.329627 31727 leader_election.cc:288] T 00000000000000000000000000000000 P 9bcdf9a29ef74caea466640f41ae92a1 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 15:05:39.329663 31732 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 9bcdf9a29ef74caea466640f41ae92a1 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 15:05:39.329790 31732 raft_consensus.cc:686] T 00000000000000000000000000000000 P 9bcdf9a29ef74caea466640f41ae92a1 [term 1 LEADER]: Becoming Leader. State: Replica: 9bcdf9a29ef74caea466640f41ae92a1, State: Running, Role: LEADER
I20260923 15:05:39.329933 31727 sys_catalog.cc:564] T 00000000000000000000000000000000 P 9bcdf9a29ef74caea466640f41ae92a1 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 15:05:39.329948 31732 consensus_queue.cc:237] T 00000000000000000000000000000000 P 9bcdf9a29ef74caea466640f41ae92a1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9bcdf9a29ef74caea466640f41ae92a1" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 34895 } }
I20260923 15:05:39.331005 31733 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9bcdf9a29ef74caea466640f41ae92a1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "9bcdf9a29ef74caea466640f41ae92a1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9bcdf9a29ef74caea466640f41ae92a1" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 34895 } } }
I20260923 15:05:39.331103 31733 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9bcdf9a29ef74caea466640f41ae92a1 [sys.catalog]: This master's current role is: LEADER
I20260923 15:05:39.331302 31734 sys_catalog.cc:455] T 00000000000000000000000000000000 P 9bcdf9a29ef74caea466640f41ae92a1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 9bcdf9a29ef74caea466640f41ae92a1. Latest consensus state: current_term: 1 leader_uuid: "9bcdf9a29ef74caea466640f41ae92a1" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "9bcdf9a29ef74caea466640f41ae92a1" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 34895 } } }
I20260923 15:05:39.331350 31734 sys_catalog.cc:458] T 00000000000000000000000000000000 P 9bcdf9a29ef74caea466640f41ae92a1 [sys.catalog]: This master's current role is: LEADER
W20260923 15:05:39.332144 31748 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 9bcdf9a29ef74caea466640f41ae92a1: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260923 15:05:39.332248 31748 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260923 15:05:39.332311 31742 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 15:05:39.332481 31742 catalog_manager.cc:1484] Initializing Kudu cluster ID...
Sep 23 15:05:39 dist-test-slave-cnb0 krb5kdc[31646](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175939, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.75.190@KRBTEST.COM
I20260923 15:05:39.334775 31742 catalog_manager.cc:1347] Generated new cluster ID: b3e3a1b8d71c459f8918a5185ef00b75
I20260923 15:05:39.334836 31742 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 15:05:39.338068 31731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:39.326774 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43234 (local address 127.13.75.190:34895)
0923 15:05:39.326966 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0923 15:05:39.326981 (+ 15us) server_negotiation.cc:399] Waiting for connection header
0923 15:05:39.327007 (+ 26us) server_negotiation.cc:407] Connection header received
0923 15:05:39.327533 (+ 526us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:05:39.327540 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:05:39.327586 (+ 46us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:05:39.327700 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:05:39.328637 (+ 937us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:05:39.330046 (+ 1409us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:05:39.331103 (+ 1057us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:05:39.331854 (+ 751us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:05:39.331963 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:05:39.334725 (+ 2762us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:05:39.334728 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:05:39.334728 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:05:39.334738 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:05:39.336524 (+ 1786us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:05:39.336912 (+ 388us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:05:39.336915 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:05:39.336915 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:05:39.337024 (+ 109us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:05:39.337254 (+ 230us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:05:39.337255 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:05:39.337255 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:05:39.337494 (+ 239us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:05:39.337680 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0923 15:05:39.337766 (+ 86us) server_negotiation.cc:299] Negotiation successful
0923 15:05:39.337901 (+ 135us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":138,"thread_start_us":77,"threads_started":1}
I20260923 15:05:39.367421 31742 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 15:05:39.368217 31742 catalog_manager.cc:1504] Loading token signing keys...
I20260923 15:05:39.373257 31742 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 9bcdf9a29ef74caea466640f41ae92a1: Generated new TSK 0
I20260923 15:05:39.373471 31742 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.75.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.75.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.75.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.129.keytab.
Entry for principal kudu/127.13.75.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.129.keytab.
I20260923 15:05:39.406697 13614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKC4soy/build/release/bin/kudu
/tmp/dist-test-taskKC4soy/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175691491353-13614-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175691491353-13614-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175691491353-13614-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175691491353-13614-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.75.129:0
--local_ip_for_outbound_sockets=127.13.75.129
--webserver_interface=127.13.75.129
--webserver_port=0
--tserver_master_addrs=127.13.75.190:34895
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.129.keytab
--principal=kudu/127.13.75.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.75.148:34427
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175691491353-13614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175691491353-13614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175691491353-13614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:05:39.488584 31753 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:05:39.488799 31753 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:05:39.488823 31753 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:05:39.489984 31753 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:05:39.490026 31753 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:05:39.490053 31753 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.75.129
I20260923 15:05:39.491724 31753 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.75.148:34427
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175691491353-13614-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175691491353-13614-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.129.keytab
--principal=kudu/127.13.75.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.75.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175691491353-13614-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.75.129
--webserver_port=0
--tserver_master_addrs=127.13.75.190:34895
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31753
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.75.129
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175691491353-13614-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:05:39.491990 31753 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:05:39.492250 31753 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:05:39.497366 31759 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:05:39.497419 31761 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:05:39.497581 31758 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:05:39.497618 31753 server_base.cc:1034] running on GCE node
I20260923 15:05:39.497856 31753 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:05:39.498111 31753 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:05:39.499315 31753 hybrid_clock.cc:648] HybridClock initialized: now 1790175939499294 us; error 31 us; skew 500 ppm
Sep 23 15:05:39 dist-test-slave-cnb0 krb5kdc[31646](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175939, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:05:39.502007 31753 init.cc:377] Logged in from keytab as kudu/127.13.75.129@KRBTEST.COM (short username kudu)
I20260923 15:05:39.502642 31753 webserver.cc:460] Webserver started at http://127.13.75.129:35263/ using document root <none> and password file <none>
I20260923 15:05:39.502964 31753 fs_manager.cc:359] Metadata directory not provided
I20260923 15:05:39.503036 31753 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:05:39.503177 31753 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:05:39.504297 31753 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175691491353-13614-0/minicluster-data/ts-0/data/instance:
uuid: "cb7001eb72124596a2779dcad5e977c4"
format_stamp: "Formatted at 2026-09-23 15:05:39 on dist-test-slave-cnb0"
server_key: "00efbc99409e653257f5a5a56ab57260"
server_key_iv: "52a722f025beed9f1d1eb92493041aac"
server_key_version: "encryptionkey@0"
I20260923 15:05:39.504668 31753 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175691491353-13614-0/minicluster-data/ts-0/wal/instance:
uuid: "cb7001eb72124596a2779dcad5e977c4"
format_stamp: "Formatted at 2026-09-23 15:05:39 on dist-test-slave-cnb0"
server_key: "00efbc99409e653257f5a5a56ab57260"
server_key_iv: "52a722f025beed9f1d1eb92493041aac"
server_key_version: "encryptionkey@0"
I20260923 15:05:39.506038 31753 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260923 15:05:39.506851 31768 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:05:39.507120 31753 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260923 15:05:39.507216 31753 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175691491353-13614-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175691491353-13614-0/minicluster-data/ts-0/wal
uuid: "cb7001eb72124596a2779dcad5e977c4"
format_stamp: "Formatted at 2026-09-23 15:05:39 on dist-test-slave-cnb0"
server_key: "00efbc99409e653257f5a5a56ab57260"
server_key_iv: "52a722f025beed9f1d1eb92493041aac"
server_key_version: "encryptionkey@0"
I20260923 15:05:39.507277 31753 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175691491353-13614-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175691491353-13614-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175691491353-13614-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:05:39.523538 31753 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:05:39.525925 31753 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:05:39.526047 31753 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:05:39.526237 31753 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 15:05:39.526584 31753 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 15:05:39.526647 31753 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 15:05:39.526691 31753 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 15:05:39.526717 31753 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 15:05:39.533036 31753 rpc_server.cc:307] RPC server started. Bound to: 127.13.75.129:36275
I20260923 15:05:39.533164 31882 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.75.129:36275 every 8 connection(s)
I20260923 15:05:39.533488 31753 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175691491353-13614-0/minicluster-data/ts-0/data/info.pb
Sep 23 15:05:39 dist-test-slave-cnb0 krb5kdc[31646](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175939, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.129@KRBTEST.COM for kudu/127.13.75.190@KRBTEST.COM
I20260923 15:05:39.542053 13614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKC4soy/build/release/bin/kudu as pid 31753
I20260923 15:05:39.542167 13614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175691491353-13614-0/minicluster-data/ts-0/wal/instance
I20260923 15:05:39.542398 13614 external_mini_cluster.cc:1428] Setting key 2ac596b36ab44f187ddf8f8f409f584a
I20260923 15:05:39.544513 31731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:39.534293 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.129:50307 (local address 127.13.75.190:34895)
0923 15:05:39.534365 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0923 15:05:39.534368 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:05:39.534584 (+ 216us) server_negotiation.cc:407] Connection header received
0923 15:05:39.535231 (+ 647us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:05:39.535232 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:05:39.535244 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:05:39.535313 (+ 69us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:05:39.536375 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:05:39.536914 (+ 539us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:05:39.537951 (+ 1037us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:05:39.538248 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:05:39.538317 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:05:39.541019 (+ 2702us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:05:39.541020 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:05:39.541020 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:05:39.541028 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:05:39.542777 (+ 1749us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:05:39.543410 (+ 633us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:05:39.543412 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:05:39.543412 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:05:39.543481 (+ 69us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:05:39.543796 (+ 315us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:05:39.543796 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:05:39.543797 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:05:39.543906 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:05:39.543955 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0923 15:05:39.544359 (+ 404us) server_negotiation.cc:299] Negotiation successful
0923 15:05:39.544445 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260923 15:05:39.544775 31885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:39.534268 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:34895 (local address 127.13.75.129:50307)
0923 15:05:39.534449 (+ 181us) negotiation.cc:107] Waiting for socket to connect
0923 15:05:39.534494 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0923 15:05:39.535065 (+ 571us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:05:39.535412 (+ 347us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:05:39.535416 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:05:39.536005 (+ 589us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:05:39.536218 (+ 213us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:05:39.536223 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:05:39.537032 (+ 809us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:05:39.537033 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:05:39.537844 (+ 811us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:05:39.537846 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:05:39.538369 (+ 523us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:05:39.538370 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:05:39.538485 (+ 115us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:05:39.539288 (+ 803us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:05:39.539297 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:05:39.540891 (+ 1594us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:05:39.542924 (+ 2033us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:05:39.542934 (+ 10us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:05:39.542936 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:05:39.543302 (+ 366us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:05:39.543562 (+ 260us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:05:39.543563 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:05:39.543563 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:05:39.543720 (+ 157us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:05:39.543968 (+ 248us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:05:39.543976 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:05:39.544163 (+ 187us) client_negotiation.cc:769] Sending connection context
0923 15:05:39.544280 (+ 117us) client_negotiation.cc:240] Negotiation successful
0923 15:05:39.544476 (+ 196us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":136,"thread_start_us":99,"threads_started":1}
I20260923 15:05:39.545305 31883 heartbeater.cc:347] Connected to a master server at 127.13.75.190:34895
I20260923 15:05:39.545400 31883 heartbeater.cc:464] Registering TS with master...
I20260923 15:05:39.545612 31883 heartbeater.cc:511] Master 127.13.75.190:34895 requested a full tablet report, sending...
I20260923 15:05:39.546216 31691 ts_manager.cc:194] Registered new tserver with Master: cb7001eb72124596a2779dcad5e977c4 (127.13.75.129:36275)
I20260923 15:05:39.547224 31691 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.75.129@KRBTEST.COM'} at 127.13.75.129:50307
WARNING: no policy specified for kudu/127.13.75.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.75.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.75.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.130.keytab.
Entry for principal kudu/127.13.75.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.130.keytab.
I20260923 15:05:39.571048 13614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKC4soy/build/release/bin/kudu
/tmp/dist-test-taskKC4soy/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175691491353-13614-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175691491353-13614-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175691491353-13614-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175691491353-13614-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.75.130:0
--local_ip_for_outbound_sockets=127.13.75.130
--webserver_interface=127.13.75.130
--webserver_port=0
--tserver_master_addrs=127.13.75.190:34895
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.130.keytab
--principal=kudu/127.13.75.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.75.148:34427
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175691491353-13614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175691491353-13614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175691491353-13614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:05:39.656368 31888 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:05:39.656577 31888 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:05:39.656602 31888 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:05:39.657802 31888 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:05:39.657850 31888 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:05:39.657877 31888 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.75.130
I20260923 15:05:39.659732 31888 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.75.148:34427
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175691491353-13614-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175691491353-13614-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.130.keytab
--principal=kudu/127.13.75.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.75.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175691491353-13614-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.75.130
--webserver_port=0
--tserver_master_addrs=127.13.75.190:34895
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.31888
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.75.130
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175691491353-13614-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:05:39.660027 31888 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:05:39.660326 31888 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:05:39.667132 31894 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:05:39.667132 31896 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:05:39.667300 31893 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:05:39.667871 31888 server_base.cc:1034] running on GCE node
I20260923 15:05:39.668115 31888 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:05:39.668429 31888 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:05:39.669652 31888 hybrid_clock.cc:648] HybridClock initialized: now 1790175939669636 us; error 36 us; skew 500 ppm
Sep 23 15:05:39 dist-test-slave-cnb0 krb5kdc[31646](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175939, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:05:39.672605 31888 init.cc:377] Logged in from keytab as kudu/127.13.75.130@KRBTEST.COM (short username kudu)
I20260923 15:05:39.673225 31888 webserver.cc:460] Webserver started at http://127.13.75.130:36379/ using document root <none> and password file <none>
I20260923 15:05:39.673491 31888 fs_manager.cc:359] Metadata directory not provided
I20260923 15:05:39.673563 31888 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:05:39.673718 31888 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:05:39.674811 31888 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175691491353-13614-0/minicluster-data/ts-1/data/instance:
uuid: "4bbff0ac238d4eb3ba3ccc0c6392d82d"
format_stamp: "Formatted at 2026-09-23 15:05:39 on dist-test-slave-cnb0"
server_key: "d8784c042fe69cacc3408c8fc82161df"
server_key_iv: "d19be4859341d02edc7365efe45bbd76"
server_key_version: "encryptionkey@0"
I20260923 15:05:39.675261 31888 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175691491353-13614-0/minicluster-data/ts-1/wal/instance:
uuid: "4bbff0ac238d4eb3ba3ccc0c6392d82d"
format_stamp: "Formatted at 2026-09-23 15:05:39 on dist-test-slave-cnb0"
server_key: "d8784c042fe69cacc3408c8fc82161df"
server_key_iv: "d19be4859341d02edc7365efe45bbd76"
server_key_version: "encryptionkey@0"
I20260923 15:05:39.676819 31888 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 15:05:39.677695 31903 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:05:39.678084 31888 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:05:39.678216 31888 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175691491353-13614-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175691491353-13614-0/minicluster-data/ts-1/wal
uuid: "4bbff0ac238d4eb3ba3ccc0c6392d82d"
format_stamp: "Formatted at 2026-09-23 15:05:39 on dist-test-slave-cnb0"
server_key: "d8784c042fe69cacc3408c8fc82161df"
server_key_iv: "d19be4859341d02edc7365efe45bbd76"
server_key_version: "encryptionkey@0"
I20260923 15:05:39.678367 31888 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175691491353-13614-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175691491353-13614-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175691491353-13614-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:05:39.695300 31888 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:05:39.697670 31888 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:05:39.697885 31888 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:05:39.698155 31888 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 15:05:39.698523 31888 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 15:05:39.698585 31888 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 15:05:39.698632 31888 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 15:05:39.698665 31888 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 15:05:39.705324 31888 rpc_server.cc:307] RPC server started. Bound to: 127.13.75.130:41113
I20260923 15:05:39.705374 32016 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.75.130:41113 every 8 connection(s)
I20260923 15:05:39.705744 31888 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175691491353-13614-0/minicluster-data/ts-1/data/info.pb
I20260923 15:05:39.706522 13614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKC4soy/build/release/bin/kudu as pid 31888
I20260923 15:05:39.706642 13614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175691491353-13614-0/minicluster-data/ts-1/wal/instance
I20260923 15:05:39.706852 13614 external_mini_cluster.cc:1428] Setting key f252662e05ccb686e96aa6a5e20b4bf5
Sep 23 15:05:39 dist-test-slave-cnb0 krb5kdc[31646](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175939, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.130@KRBTEST.COM for kudu/127.13.75.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.13.75.131@KRBTEST.COM; defaulting to no policy
I20260923 15:05:39.716921 31731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:39.706609 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.130:41371 (local address 127.13.75.190:34895)
0923 15:05:39.706672 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0923 15:05:39.706674 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:05:39.706895 (+ 221us) server_negotiation.cc:407] Connection header received
0923 15:05:39.707535 (+ 640us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:05:39.707536 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:05:39.707545 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:05:39.707632 (+ 87us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:05:39.708648 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:05:39.709173 (+ 525us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:05:39.710338 (+ 1165us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:05:39.710538 (+ 200us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:05:39.710629 (+ 91us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:05:39.713279 (+ 2650us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:05:39.713281 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:05:39.713281 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:05:39.713292 (+ 11us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:05:39.715077 (+ 1785us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:05:39.715719 (+ 642us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:05:39.715720 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:05:39.715721 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:05:39.715795 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:05:39.716131 (+ 336us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:05:39.716133 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:05:39.716133 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:05:39.716274 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:05:39.716352 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0923 15:05:39.716719 (+ 367us) server_negotiation.cc:299] Negotiation successful
0923 15:05:39.716823 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260923 15:05:39.717259 32019 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:39.706560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:34895 (local address 127.13.75.130:41371)
0923 15:05:39.706756 (+ 196us) negotiation.cc:107] Waiting for socket to connect
0923 15:05:39.706802 (+ 46us) client_negotiation.cc:174] Beginning negotiation
0923 15:05:39.707430 (+ 628us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:05:39.707731 (+ 301us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:05:39.707735 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:05:39.708332 (+ 597us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:05:39.708538 (+ 206us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:05:39.708543 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:05:39.709286 (+ 743us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:05:39.709288 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:05:39.710246 (+ 958us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:05:39.710249 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:05:39.710642 (+ 393us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:05:39.710643 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:05:39.710780 (+ 137us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:05:39.711478 (+ 698us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:05:39.711487 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:05:39.713146 (+ 1659us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:05:39.715333 (+ 2187us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:05:39.715345 (+ 12us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:05:39.715348 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:05:39.715625 (+ 277us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:05:39.715875 (+ 250us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:05:39.715876 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:05:39.715876 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:05:39.716046 (+ 170us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:05:39.716353 (+ 307us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:05:39.716363 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:05:39.716553 (+ 190us) client_negotiation.cc:769] Sending connection context
0923 15:05:39.716680 (+ 127us) client_negotiation.cc:240] Negotiation successful
0923 15:05:39.716875 (+ 195us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":153,"thread_start_us":88,"threads_started":1}
I20260923 15:05:39.717770 32017 heartbeater.cc:347] Connected to a master server at 127.13.75.190:34895
I20260923 15:05:39.717923 32017 heartbeater.cc:464] Registering TS with master...
I20260923 15:05:39.718143 32017 heartbeater.cc:511] Master 127.13.75.190:34895 requested a full tablet report, sending...
I20260923 15:05:39.718736 31691 ts_manager.cc:194] Registered new tserver with Master: 4bbff0ac238d4eb3ba3ccc0c6392d82d (127.13.75.130:41113)
I20260923 15:05:39.719265 31691 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.75.130@KRBTEST.COM'} at 127.13.75.130:41371
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.75.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.75.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.131.keytab.
Entry for principal kudu/127.13.75.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.131.keytab.
I20260923 15:05:39.736213 13614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKC4soy/build/release/bin/kudu
/tmp/dist-test-taskKC4soy/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175691491353-13614-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175691491353-13614-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175691491353-13614-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175691491353-13614-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.75.131:0
--local_ip_for_outbound_sockets=127.13.75.131
--webserver_interface=127.13.75.131
--webserver_port=0
--tserver_master_addrs=127.13.75.190:34895
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.131.keytab
--principal=kudu/127.13.75.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.75.148:34427
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175691491353-13614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175691491353-13614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175691491353-13614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:05:39.821501 32022 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:05:39.821717 32022 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:05:39.821743 32022 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:05:39.823021 32022 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:05:39.823071 32022 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:05:39.823100 32022 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.75.131
I20260923 15:05:39.824641 32022 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.75.148:34427
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175691491353-13614-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175691491353-13614-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.131.keytab
--principal=kudu/127.13.75.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.75.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175691491353-13614-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.75.131
--webserver_port=0
--tserver_master_addrs=127.13.75.190:34895
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32022
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.75.131
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175691491353-13614-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:05:39.824904 32022 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:05:39.825160 32022 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:05:39.830791 32028 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:05:39.830785 32027 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:05:39.831035 32030 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:05:39.831331 32022 server_base.cc:1034] running on GCE node
I20260923 15:05:39.831584 32022 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:05:39.831976 32022 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:05:39.833154 32022 hybrid_clock.cc:648] HybridClock initialized: now 1790175939833150 us; error 41 us; skew 500 ppm
Sep 23 15:05:39 dist-test-slave-cnb0 krb5kdc[31646](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175939, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:05:39.836002 32022 init.cc:377] Logged in from keytab as kudu/127.13.75.131@KRBTEST.COM (short username kudu)
I20260923 15:05:39.836613 32022 webserver.cc:460] Webserver started at http://127.13.75.131:43537/ using document root <none> and password file <none>
I20260923 15:05:39.836903 32022 fs_manager.cc:359] Metadata directory not provided
I20260923 15:05:39.836983 32022 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:05:39.837152 32022 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:05:39.838393 32022 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175691491353-13614-0/minicluster-data/ts-2/data/instance:
uuid: "1c51cd4531f2488992d5568d0129709c"
format_stamp: "Formatted at 2026-09-23 15:05:39 on dist-test-slave-cnb0"
server_key: "8489f508e3e96b697019a3c9e5630fe2"
server_key_iv: "4cc696071968425e62d3afd34dda4899"
server_key_version: "encryptionkey@0"
I20260923 15:05:39.838814 32022 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175691491353-13614-0/minicluster-data/ts-2/wal/instance:
uuid: "1c51cd4531f2488992d5568d0129709c"
format_stamp: "Formatted at 2026-09-23 15:05:39 on dist-test-slave-cnb0"
server_key: "8489f508e3e96b697019a3c9e5630fe2"
server_key_iv: "4cc696071968425e62d3afd34dda4899"
server_key_version: "encryptionkey@0"
I20260923 15:05:39.840435 32022 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:05:39.841519 32037 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:05:39.841756 32022 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:05:39.841887 32022 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175691491353-13614-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175691491353-13614-0/minicluster-data/ts-2/wal
uuid: "1c51cd4531f2488992d5568d0129709c"
format_stamp: "Formatted at 2026-09-23 15:05:39 on dist-test-slave-cnb0"
server_key: "8489f508e3e96b697019a3c9e5630fe2"
server_key_iv: "4cc696071968425e62d3afd34dda4899"
server_key_version: "encryptionkey@0"
I20260923 15:05:39.841981 32022 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175691491353-13614-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175691491353-13614-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175691491353-13614-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:05:39.865206 32022 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:05:39.867398 32022 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:05:39.867575 32022 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:05:39.867831 32022 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 15:05:39.868160 32022 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 15:05:39.868214 32022 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 15:05:39.868257 32022 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 15:05:39.868314 32022 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 15:05:39.874567 32022 rpc_server.cc:307] RPC server started. Bound to: 127.13.75.131:34795
I20260923 15:05:39.874622 32150 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.75.131:34795 every 8 connection(s)
I20260923 15:05:39.875072 32022 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175691491353-13614-0/minicluster-data/ts-2/data/info.pb
Sep 23 15:05:39 dist-test-slave-cnb0 krb5kdc[31646](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175939, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.131@KRBTEST.COM for kudu/127.13.75.190@KRBTEST.COM
I20260923 15:05:39.881882 13614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKC4soy/build/release/bin/kudu as pid 32022
I20260923 15:05:39.881968 13614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175691491353-13614-0/minicluster-data/ts-2/wal/instance
I20260923 15:05:39.882148 13614 external_mini_cluster.cc:1428] Setting key aea3df22c9c341435a3389e3cf4925c8
I20260923 15:05:39.885066 31731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:39.875909 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.131:43987 (local address 127.13.75.190:34895)
0923 15:05:39.876023 (+ 114us) server_negotiation.cc:206] Beginning negotiation
0923 15:05:39.876025 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:05:39.876279 (+ 254us) server_negotiation.cc:407] Connection header received
0923 15:05:39.876853 (+ 574us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:05:39.876854 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:05:39.876867 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:05:39.876925 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:05:39.877903 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:05:39.878420 (+ 517us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:05:39.879475 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:05:39.879695 (+ 220us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:05:39.879781 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:05:39.882038 (+ 2257us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:05:39.882040 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:05:39.882040 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:05:39.882056 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:05:39.883808 (+ 1752us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:05:39.884199 (+ 391us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:05:39.884200 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:05:39.884201 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:05:39.884297 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:05:39.884523 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:05:39.884523 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:05:39.884524 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:05:39.884633 (+ 109us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:05:39.884686 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0923 15:05:39.884933 (+ 247us) server_negotiation.cc:299] Negotiation successful
0923 15:05:39.885002 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":51}
I20260923 15:05:39.885215 32153 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:39.875868 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:34895 (local address 127.13.75.131:43987)
0923 15:05:39.876084 (+ 216us) negotiation.cc:107] Waiting for socket to connect
0923 15:05:39.876177 (+ 93us) client_negotiation.cc:174] Beginning negotiation
0923 15:05:39.876741 (+ 564us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:05:39.877004 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:05:39.877007 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:05:39.877598 (+ 591us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:05:39.877801 (+ 203us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:05:39.877805 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:05:39.878518 (+ 713us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:05:39.878519 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:05:39.879383 (+ 864us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:05:39.879385 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:05:39.879795 (+ 410us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:05:39.879796 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:05:39.879919 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:05:39.880407 (+ 488us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:05:39.880438 (+ 31us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:05:39.881901 (+ 1463us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:05:39.883872 (+ 1971us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:05:39.883877 (+ 5us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:05:39.883879 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:05:39.884097 (+ 218us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:05:39.884366 (+ 269us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:05:39.884366 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:05:39.884367 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:05:39.884463 (+ 96us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:05:39.884689 (+ 226us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:05:39.884693 (+ 4us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:05:39.884821 (+ 128us) client_negotiation.cc:769] Sending connection context
0923 15:05:39.884904 (+ 83us) client_negotiation.cc:240] Negotiation successful
0923 15:05:39.885024 (+ 120us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":87,"threads_started":1}
I20260923 15:05:39.885725 32151 heartbeater.cc:347] Connected to a master server at 127.13.75.190:34895
I20260923 15:05:39.885828 32151 heartbeater.cc:464] Registering TS with master...
I20260923 15:05:39.886009 32151 heartbeater.cc:511] Master 127.13.75.190:34895 requested a full tablet report, sending...
I20260923 15:05:39.886387 31691 ts_manager.cc:194] Registered new tserver with Master: 1c51cd4531f2488992d5568d0129709c (127.13.75.131:34795)
I20260923 15:05:39.886792 31691 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.75.131@KRBTEST.COM'} at 127.13.75.131:43987
I20260923 15:05:39.894822 13614 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260923 15:05:39.902488 31731 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:39.896157 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:43238 (local address 127.13.75.190:34895)
0923 15:05:39.896220 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0923 15:05:39.896222 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:05:39.896430 (+ 208us) server_negotiation.cc:407] Connection header received
0923 15:05:39.896536 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:05:39.896537 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:05:39.896548 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:05:39.896607 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:05:39.897256 (+ 649us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:05:39.897742 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:05:39.898449 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:05:39.898641 (+ 192us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:05:39.898727 (+ 86us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:05:39.900078 (+ 1351us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:05:39.900079 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:05:39.900080 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:05:39.900087 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:05:39.901241 (+ 1154us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:05:39.901630 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:05:39.901631 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:05:39.901632 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:05:39.901692 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:05:39.901913 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:05:39.901914 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:05:39.901914 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:05:39.902057 (+ 143us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:05:39.902173 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0923 15:05:39.902319 (+ 146us) server_negotiation.cc:299] Negotiation successful
0923 15:05:39.902388 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260923 15:05:39.904301 31691 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43238:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 15:05:39.905911 31691 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 15:05:39.912560 32165 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:39.908532 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.130:41113 (local address 127.0.0.1:38476)
0923 15:05:39.908742 (+ 210us) negotiation.cc:107] Waiting for socket to connect
0923 15:05:39.908753 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 15:05:39.908797 (+ 44us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:05:39.909314 (+ 517us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:05:39.909315 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:05:39.909317 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:05:39.909350 (+ 33us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:05:39.909351 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:05:39.910588 (+ 1237us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:05:39.910589 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:05:39.911465 (+ 876us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:05:39.911468 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:05:39.912236 (+ 768us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:05:39.912237 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:05:39.912312 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:05:39.912327 (+ 15us) client_negotiation.cc:769] Sending connection context
0923 15:05:39.912407 (+ 80us) client_negotiation.cc:240] Negotiation successful
0923 15:05:39.912474 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":66,"threads_started":1}
I20260923 15:05:39.912561 32167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:39.908722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38476 (local address 127.13.75.130:41113)
0923 15:05:39.908921 (+ 199us) server_negotiation.cc:206] Beginning negotiation
0923 15:05:39.908925 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 15:05:39.908932 (+ 7us) server_negotiation.cc:407] Connection header received
0923 15:05:39.908963 (+ 31us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:05:39.908964 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:05:39.908976 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:05:39.909234 (+ 258us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:05:39.909426 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:05:39.910485 (+ 1059us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:05:39.911556 (+ 1071us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:05:39.912147 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:05:39.912224 (+ 77us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:05:39.912284 (+ 60us) server_negotiation.cc:1035] Waiting for connection context
0923 15:05:39.912431 (+ 147us) server_negotiation.cc:299] Negotiation successful
0923 15:05:39.912472 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":165,"thread_start_us":94,"threads_started":1}
I20260923 15:05:39.912683 32163 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:39.908347 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.129:36275 (local address 127.0.0.1:46190)
0923 15:05:39.908485 (+ 138us) negotiation.cc:107] Waiting for socket to connect
0923 15:05:39.908526 (+ 41us) client_negotiation.cc:174] Beginning negotiation
0923 15:05:39.908591 (+ 65us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:05:39.909040 (+ 449us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:05:39.909043 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:05:39.909046 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:05:39.909131 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:05:39.909133 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:05:39.910534 (+ 1401us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:05:39.910535 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:05:39.911645 (+ 1110us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:05:39.911648 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:05:39.912477 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:05:39.912478 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:05:39.912545 (+ 67us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:05:39.912548 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 15:05:39.912578 (+ 30us) client_negotiation.cc:240] Negotiation successful
0923 15:05:39.912616 (+ 38us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":115,"thread_start_us":92,"threads_started":1}
I20260923 15:05:39.912997 32162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:39.908324 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.131:34795 (local address 127.0.0.1:42212)
0923 15:05:39.908482 (+ 158us) negotiation.cc:107] Waiting for socket to connect
0923 15:05:39.908525 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0923 15:05:39.908591 (+ 66us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:05:39.909072 (+ 481us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:05:39.909073 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:05:39.909076 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:05:39.909130 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:05:39.909131 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:05:39.910277 (+ 1146us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:05:39.910278 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:05:39.911891 (+ 1613us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:05:39.911894 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:05:39.912781 (+ 887us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:05:39.912782 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:05:39.912836 (+ 54us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:05:39.912839 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 15:05:39.912904 (+ 65us) client_negotiation.cc:240] Negotiation successful
0923 15:05:39.912938 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":53,"threads_started":1}
I20260923 15:05:39.913007 32166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:39.908671 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46190 (local address 127.13.75.129:36275)
0923 15:05:39.908852 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0923 15:05:39.908858 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 15:05:39.908871 (+ 13us) server_negotiation.cc:407] Connection header received
0923 15:05:39.908913 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:05:39.908915 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:05:39.908933 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:05:39.909020 (+ 87us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:05:39.909224 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:05:39.910426 (+ 1202us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:05:39.911733 (+ 1307us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:05:39.912388 (+ 655us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:05:39.912786 (+ 398us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:05:39.912855 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0923 15:05:39.912881 (+ 26us) server_negotiation.cc:299] Negotiation successful
0923 15:05:39.912925 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":148,"thread_start_us":114,"threads_started":1}
I20260923 15:05:39.913039 32164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:39.908459 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:42212 (local address 127.13.75.131:34795)
0923 15:05:39.908908 (+ 449us) server_negotiation.cc:206] Beginning negotiation
0923 15:05:39.908911 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:05:39.908920 (+ 9us) server_negotiation.cc:407] Connection header received
0923 15:05:39.908957 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:05:39.908959 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:05:39.908973 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:05:39.909204 (+ 231us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:05:39.909275 (+ 71us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:05:39.910171 (+ 896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:05:39.911977 (+ 1806us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:05:39.912692 (+ 715us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:05:39.912774 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:05:39.912833 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0923 15:05:39.912918 (+ 85us) server_negotiation.cc:299] Negotiation successful
0923 15:05:39.912968 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":418,"thread_start_us":87,"threads_started":1}
I20260923 15:05:39.913801 32085 tablet_service.cc:1467] Processing CreateTablet for tablet 2994cb9a8f2d473aa26ca15513cb2ad3 (DEFAULT_TABLE table=test-table [id=99bc1758cbe7470495b29cd64d2000ba]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 15:05:39.913829 31816 tablet_service.cc:1467] Processing CreateTablet for tablet 2994cb9a8f2d473aa26ca15513cb2ad3 (DEFAULT_TABLE table=test-table [id=99bc1758cbe7470495b29cd64d2000ba]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 15:05:39.913864 31951 tablet_service.cc:1467] Processing CreateTablet for tablet 2994cb9a8f2d473aa26ca15513cb2ad3 (DEFAULT_TABLE table=test-table [id=99bc1758cbe7470495b29cd64d2000ba]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 15:05:39.914151 32085 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2994cb9a8f2d473aa26ca15513cb2ad3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:05:39.914151 31951 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2994cb9a8f2d473aa26ca15513cb2ad3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:05:39.914151 31816 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 2994cb9a8f2d473aa26ca15513cb2ad3. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:05:39.917029 32168 tablet_bootstrap.cc:492] T 2994cb9a8f2d473aa26ca15513cb2ad3 P 1c51cd4531f2488992d5568d0129709c: Bootstrap starting.
I20260923 15:05:39.917521 32169 tablet_bootstrap.cc:492] T 2994cb9a8f2d473aa26ca15513cb2ad3 P 4bbff0ac238d4eb3ba3ccc0c6392d82d: Bootstrap starting.
I20260923 15:05:39.918143 32168 tablet_bootstrap.cc:654] T 2994cb9a8f2d473aa26ca15513cb2ad3 P 1c51cd4531f2488992d5568d0129709c: Neither blocks nor log segments found. Creating new log.
I20260923 15:05:39.918313 32169 tablet_bootstrap.cc:654] T 2994cb9a8f2d473aa26ca15513cb2ad3 P 4bbff0ac238d4eb3ba3ccc0c6392d82d: Neither blocks nor log segments found. Creating new log.
I20260923 15:05:39.918805 32168 log.cc:824] T 2994cb9a8f2d473aa26ca15513cb2ad3 P 1c51cd4531f2488992d5568d0129709c: Log is configured to *not* fsync() on all Append() calls
I20260923 15:05:39.918974 32169 log.cc:824] T 2994cb9a8f2d473aa26ca15513cb2ad3 P 4bbff0ac238d4eb3ba3ccc0c6392d82d: Log is configured to *not* fsync() on all Append() calls
I20260923 15:05:39.919147 32170 tablet_bootstrap.cc:492] T 2994cb9a8f2d473aa26ca15513cb2ad3 P cb7001eb72124596a2779dcad5e977c4: Bootstrap starting.
I20260923 15:05:39.919963 32169 tablet_bootstrap.cc:492] T 2994cb9a8f2d473aa26ca15513cb2ad3 P 4bbff0ac238d4eb3ba3ccc0c6392d82d: No bootstrap required, opened a new log
I20260923 15:05:39.919970 32168 tablet_bootstrap.cc:492] T 2994cb9a8f2d473aa26ca15513cb2ad3 P 1c51cd4531f2488992d5568d0129709c: No bootstrap required, opened a new log
I20260923 15:05:39.919963 32170 tablet_bootstrap.cc:654] T 2994cb9a8f2d473aa26ca15513cb2ad3 P cb7001eb72124596a2779dcad5e977c4: Neither blocks nor log segments found. Creating new log.
I20260923 15:05:39.920040 32169 ts_tablet_manager.cc:1397] T 2994cb9a8f2d473aa26ca15513cb2ad3 P 4bbff0ac238d4eb3ba3ccc0c6392d82d: Time spent bootstrapping tablet: real 0.003s user 0.000s sys 0.002s
I20260923 15:05:39.920040 32168 ts_tablet_manager.cc:1397] T 2994cb9a8f2d473aa26ca15513cb2ad3 P 1c51cd4531f2488992d5568d0129709c: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260923 15:05:39.920518 32170 log.cc:824] T 2994cb9a8f2d473aa26ca15513cb2ad3 P cb7001eb72124596a2779dcad5e977c4: Log is configured to *not* fsync() on all Append() calls
I20260923 15:05:39.921310 32170 tablet_bootstrap.cc:492] T 2994cb9a8f2d473aa26ca15513cb2ad3 P cb7001eb72124596a2779dcad5e977c4: No bootstrap required, opened a new log
I20260923 15:05:39.921407 32170 ts_tablet_manager.cc:1397] T 2994cb9a8f2d473aa26ca15513cb2ad3 P cb7001eb72124596a2779dcad5e977c4: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260923 15:05:39.922127 32168 raft_consensus.cc:348] T 2994cb9a8f2d473aa26ca15513cb2ad3 P 1c51cd4531f2488992d5568d0129709c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c51cd4531f2488992d5568d0129709c" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 34795 } } peers { permanent_uuid: "cb7001eb72124596a2779dcad5e977c4" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 36275 } } peers { permanent_uuid: "4bbff0ac238d4eb3ba3ccc0c6392d82d" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 41113 } }
I20260923 15:05:39.922127 32169 raft_consensus.cc:348] T 2994cb9a8f2d473aa26ca15513cb2ad3 P 4bbff0ac238d4eb3ba3ccc0c6392d82d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c51cd4531f2488992d5568d0129709c" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 34795 } } peers { permanent_uuid: "cb7001eb72124596a2779dcad5e977c4" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 36275 } } peers { permanent_uuid: "4bbff0ac238d4eb3ba3ccc0c6392d82d" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 41113 } }
I20260923 15:05:39.922324 32169 raft_consensus.cc:374] T 2994cb9a8f2d473aa26ca15513cb2ad3 P 4bbff0ac238d4eb3ba3ccc0c6392d82d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:05:39.922324 32168 raft_consensus.cc:374] T 2994cb9a8f2d473aa26ca15513cb2ad3 P 1c51cd4531f2488992d5568d0129709c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:05:39.922363 32168 raft_consensus.cc:729] T 2994cb9a8f2d473aa26ca15513cb2ad3 P 1c51cd4531f2488992d5568d0129709c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 1c51cd4531f2488992d5568d0129709c, State: Initialized, Role: FOLLOWER
I20260923 15:05:39.922363 32169 raft_consensus.cc:729] T 2994cb9a8f2d473aa26ca15513cb2ad3 P 4bbff0ac238d4eb3ba3ccc0c6392d82d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 4bbff0ac238d4eb3ba3ccc0c6392d82d, State: Initialized, Role: FOLLOWER
I20260923 15:05:39.922485 32169 consensus_queue.cc:260] T 2994cb9a8f2d473aa26ca15513cb2ad3 P 4bbff0ac238d4eb3ba3ccc0c6392d82d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c51cd4531f2488992d5568d0129709c" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 34795 } } peers { permanent_uuid: "cb7001eb72124596a2779dcad5e977c4" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 36275 } } peers { permanent_uuid: "4bbff0ac238d4eb3ba3ccc0c6392d82d" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 41113 } }
I20260923 15:05:39.922485 32168 consensus_queue.cc:260] T 2994cb9a8f2d473aa26ca15513cb2ad3 P 1c51cd4531f2488992d5568d0129709c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c51cd4531f2488992d5568d0129709c" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 34795 } } peers { permanent_uuid: "cb7001eb72124596a2779dcad5e977c4" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 36275 } } peers { permanent_uuid: "4bbff0ac238d4eb3ba3ccc0c6392d82d" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 41113 } }
I20260923 15:05:39.922794 32168 ts_tablet_manager.cc:1428] T 2994cb9a8f2d473aa26ca15513cb2ad3 P 1c51cd4531f2488992d5568d0129709c: Time spent starting tablet: real 0.003s user 0.002s sys 0.000s
I20260923 15:05:39.922794 32169 ts_tablet_manager.cc:1428] T 2994cb9a8f2d473aa26ca15513cb2ad3 P 4bbff0ac238d4eb3ba3ccc0c6392d82d: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260923 15:05:39.922830 32017 heartbeater.cc:503] Master 127.13.75.190:34895 was elected leader, sending a full tablet report...
I20260923 15:05:39.923132 32151 heartbeater.cc:503] Master 127.13.75.190:34895 was elected leader, sending a full tablet report...
I20260923 15:05:39.924245 32170 raft_consensus.cc:348] T 2994cb9a8f2d473aa26ca15513cb2ad3 P cb7001eb72124596a2779dcad5e977c4 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c51cd4531f2488992d5568d0129709c" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 34795 } } peers { permanent_uuid: "cb7001eb72124596a2779dcad5e977c4" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 36275 } } peers { permanent_uuid: "4bbff0ac238d4eb3ba3ccc0c6392d82d" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 41113 } }
I20260923 15:05:39.924464 32170 raft_consensus.cc:374] T 2994cb9a8f2d473aa26ca15513cb2ad3 P cb7001eb72124596a2779dcad5e977c4 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:05:39.924523 32170 raft_consensus.cc:729] T 2994cb9a8f2d473aa26ca15513cb2ad3 P cb7001eb72124596a2779dcad5e977c4 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: cb7001eb72124596a2779dcad5e977c4, State: Initialized, Role: FOLLOWER
I20260923 15:05:39.924669 32170 consensus_queue.cc:260] T 2994cb9a8f2d473aa26ca15513cb2ad3 P cb7001eb72124596a2779dcad5e977c4 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c51cd4531f2488992d5568d0129709c" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 34795 } } peers { permanent_uuid: "cb7001eb72124596a2779dcad5e977c4" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 36275 } } peers { permanent_uuid: "4bbff0ac238d4eb3ba3ccc0c6392d82d" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 41113 } }
I20260923 15:05:39.924991 32170 ts_tablet_manager.cc:1428] T 2994cb9a8f2d473aa26ca15513cb2ad3 P cb7001eb72124596a2779dcad5e977c4: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260923 15:05:39.925051 31883 heartbeater.cc:503] Master 127.13.75.190:34895 was elected leader, sending a full tablet report...
W20260923 15:05:39.956473 32018 tablet.cc:2369] T 2994cb9a8f2d473aa26ca15513cb2ad3 P 4bbff0ac238d4eb3ba3ccc0c6392d82d: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 15:05:40.006595 32176 raft_consensus.cc:482] T 2994cb9a8f2d473aa26ca15513cb2ad3 P cb7001eb72124596a2779dcad5e977c4 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 15:05:40.006744 32176 raft_consensus.cc:504] T 2994cb9a8f2d473aa26ca15513cb2ad3 P cb7001eb72124596a2779dcad5e977c4 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c51cd4531f2488992d5568d0129709c" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 34795 } } peers { permanent_uuid: "cb7001eb72124596a2779dcad5e977c4" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 36275 } } peers { permanent_uuid: "4bbff0ac238d4eb3ba3ccc0c6392d82d" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 41113 } }
I20260923 15:05:40.007332 32176 leader_election.cc:288] T 2994cb9a8f2d473aa26ca15513cb2ad3 P cb7001eb72124596a2779dcad5e977c4 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 1c51cd4531f2488992d5568d0129709c (127.13.75.131:34795), 4bbff0ac238d4eb3ba3ccc0c6392d82d (127.13.75.130:41113)
I20260923 15:05:40.011464 32167 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:40.007652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.129:34779 (local address 127.13.75.130:41113)
0923 15:05:40.007725 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0923 15:05:40.007727 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:05:40.007737 (+ 10us) server_negotiation.cc:407] Connection header received
0923 15:05:40.007778 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:05:40.007779 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:05:40.007789 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:05:40.007846 (+ 57us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:05:40.008033 (+ 187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:05:40.008826 (+ 793us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:05:40.010270 (+ 1444us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:05:40.011035 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:05:40.011125 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:05:40.011136 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0923 15:05:40.011334 (+ 198us) server_negotiation.cc:299] Negotiation successful
0923 15:05:40.011383 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260923 15:05:40.011462 32164 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:40.007674 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.129:46755 (local address 127.13.75.131:34795)
0923 15:05:40.007733 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0923 15:05:40.007735 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:05:40.007977 (+ 242us) server_negotiation.cc:407] Connection header received
0923 15:05:40.008048 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:05:40.008049 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:05:40.008061 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:05:40.008110 (+ 49us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:05:40.008312 (+ 202us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:05:40.008976 (+ 664us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:05:40.010269 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:05:40.011036 (+ 767us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:05:40.011176 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:05:40.011189 (+ 13us) server_negotiation.cc:1035] Waiting for connection context
0923 15:05:40.011334 (+ 145us) server_negotiation.cc:299] Negotiation successful
0923 15:05:40.011386 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260923 15:05:40.011457 31885 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:40.007559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.130:41113 (local address 127.13.75.129:34779)
0923 15:05:40.007605 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0923 15:05:40.007617 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 15:05:40.007666 (+ 49us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:05:40.007874 (+ 208us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:05:40.007876 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:05:40.007878 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:05:40.007953 (+ 75us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:05:40.007955 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:05:40.008915 (+ 960us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:05:40.008916 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:05:40.010149 (+ 1233us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:05:40.010153 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:05:40.011129 (+ 976us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:05:40.011130 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:05:40.011224 (+ 94us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:05:40.011229 (+ 5us) client_negotiation.cc:769] Sending connection context
0923 15:05:40.011296 (+ 67us) client_negotiation.cc:240] Negotiation successful
0923 15:05:40.011350 (+ 54us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":22,"mutex_wait_us":1}
I20260923 15:05:40.011463 32177 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:40.007560 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.131:34795 (local address 127.13.75.129:46755)
0923 15:05:40.007883 (+ 323us) negotiation.cc:107] Waiting for socket to connect
0923 15:05:40.007894 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 15:05:40.007973 (+ 79us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:05:40.008125 (+ 152us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:05:40.008126 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:05:40.008127 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:05:40.008175 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:05:40.008175 (+ 0us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:05:40.009044 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:05:40.009045 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:05:40.010168 (+ 1123us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:05:40.010170 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:05:40.011129 (+ 959us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:05:40.011130 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:05:40.011257 (+ 127us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:05:40.011261 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 15:05:40.011314 (+ 53us) client_negotiation.cc:240] Negotiation successful
0923 15:05:40.011350 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":309,"mutex_wait_us":189,"thread_start_us":67,"threads_started":1}
I20260923 15:05:40.011942 31971 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2994cb9a8f2d473aa26ca15513cb2ad3" candidate_uuid: "cb7001eb72124596a2779dcad5e977c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4bbff0ac238d4eb3ba3ccc0c6392d82d" is_pre_election: true
I20260923 15:05:40.012138 31971 raft_consensus.cc:2454] T 2994cb9a8f2d473aa26ca15513cb2ad3 P 4bbff0ac238d4eb3ba3ccc0c6392d82d [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb7001eb72124596a2779dcad5e977c4 in term 0.
I20260923 15:05:40.012187 32105 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2994cb9a8f2d473aa26ca15513cb2ad3" candidate_uuid: "cb7001eb72124596a2779dcad5e977c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c51cd4531f2488992d5568d0129709c" is_pre_election: true
I20260923 15:05:40.012357 32105 raft_consensus.cc:2454] T 2994cb9a8f2d473aa26ca15513cb2ad3 P 1c51cd4531f2488992d5568d0129709c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate cb7001eb72124596a2779dcad5e977c4 in term 0.
I20260923 15:05:40.012380 31769 leader_election.cc:302] T 2994cb9a8f2d473aa26ca15513cb2ad3 P cb7001eb72124596a2779dcad5e977c4 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4bbff0ac238d4eb3ba3ccc0c6392d82d, cb7001eb72124596a2779dcad5e977c4; no voters:
I20260923 15:05:40.012531 32176 raft_consensus.cc:2781] T 2994cb9a8f2d473aa26ca15513cb2ad3 P cb7001eb72124596a2779dcad5e977c4 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 15:05:40.012600 32176 raft_consensus.cc:482] T 2994cb9a8f2d473aa26ca15513cb2ad3 P cb7001eb72124596a2779dcad5e977c4 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 15:05:40.012633 32176 raft_consensus.cc:3037] T 2994cb9a8f2d473aa26ca15513cb2ad3 P cb7001eb72124596a2779dcad5e977c4 [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:05:40.013373 32176 raft_consensus.cc:504] T 2994cb9a8f2d473aa26ca15513cb2ad3 P cb7001eb72124596a2779dcad5e977c4 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c51cd4531f2488992d5568d0129709c" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 34795 } } peers { permanent_uuid: "cb7001eb72124596a2779dcad5e977c4" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 36275 } } peers { permanent_uuid: "4bbff0ac238d4eb3ba3ccc0c6392d82d" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 41113 } }
I20260923 15:05:40.013532 32176 leader_election.cc:288] T 2994cb9a8f2d473aa26ca15513cb2ad3 P cb7001eb72124596a2779dcad5e977c4 [CANDIDATE]: Term 1 election: Requested vote from peers 1c51cd4531f2488992d5568d0129709c (127.13.75.131:34795), 4bbff0ac238d4eb3ba3ccc0c6392d82d (127.13.75.130:41113)
I20260923 15:05:40.013712 32105 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2994cb9a8f2d473aa26ca15513cb2ad3" candidate_uuid: "cb7001eb72124596a2779dcad5e977c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "1c51cd4531f2488992d5568d0129709c"
I20260923 15:05:40.013712 31971 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "2994cb9a8f2d473aa26ca15513cb2ad3" candidate_uuid: "cb7001eb72124596a2779dcad5e977c4" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "4bbff0ac238d4eb3ba3ccc0c6392d82d"
I20260923 15:05:40.013811 31971 raft_consensus.cc:3037] T 2994cb9a8f2d473aa26ca15513cb2ad3 P 4bbff0ac238d4eb3ba3ccc0c6392d82d [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:05:40.013811 32105 raft_consensus.cc:3037] T 2994cb9a8f2d473aa26ca15513cb2ad3 P 1c51cd4531f2488992d5568d0129709c [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:05:40.014909 31971 raft_consensus.cc:2454] T 2994cb9a8f2d473aa26ca15513cb2ad3 P 4bbff0ac238d4eb3ba3ccc0c6392d82d [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb7001eb72124596a2779dcad5e977c4 in term 1.
I20260923 15:05:40.014907 32105 raft_consensus.cc:2454] T 2994cb9a8f2d473aa26ca15513cb2ad3 P 1c51cd4531f2488992d5568d0129709c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate cb7001eb72124596a2779dcad5e977c4 in term 1.
I20260923 15:05:40.015128 31769 leader_election.cc:302] T 2994cb9a8f2d473aa26ca15513cb2ad3 P cb7001eb72124596a2779dcad5e977c4 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 4bbff0ac238d4eb3ba3ccc0c6392d82d, cb7001eb72124596a2779dcad5e977c4; no voters:
I20260923 15:05:40.015239 32176 raft_consensus.cc:2781] T 2994cb9a8f2d473aa26ca15513cb2ad3 P cb7001eb72124596a2779dcad5e977c4 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 15:05:40.015432 32176 raft_consensus.cc:686] T 2994cb9a8f2d473aa26ca15513cb2ad3 P cb7001eb72124596a2779dcad5e977c4 [term 1 LEADER]: Becoming Leader. State: Replica: cb7001eb72124596a2779dcad5e977c4, State: Running, Role: LEADER
I20260923 15:05:40.015540 32176 consensus_queue.cc:237] T 2994cb9a8f2d473aa26ca15513cb2ad3 P cb7001eb72124596a2779dcad5e977c4 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c51cd4531f2488992d5568d0129709c" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 34795 } } peers { permanent_uuid: "cb7001eb72124596a2779dcad5e977c4" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 36275 } } peers { permanent_uuid: "4bbff0ac238d4eb3ba3ccc0c6392d82d" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 41113 } }
I20260923 15:05:40.016491 31691 catalog_manager.cc:5495] T 2994cb9a8f2d473aa26ca15513cb2ad3 P cb7001eb72124596a2779dcad5e977c4 reported cstate change: term changed from 0 to 1, leader changed from <none> to cb7001eb72124596a2779dcad5e977c4 (127.13.75.129). New cstate: current_term: 1 leader_uuid: "cb7001eb72124596a2779dcad5e977c4" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "1c51cd4531f2488992d5568d0129709c" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 34795 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "cb7001eb72124596a2779dcad5e977c4" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 36275 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "4bbff0ac238d4eb3ba3ccc0c6392d82d" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 41113 } health_report { overall_health: UNKNOWN } } }
I20260923 15:05:40.030061 32166 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:40.027590 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46206 (local address 127.13.75.129:36275)
0923 15:05:40.027656 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0923 15:05:40.027658 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:05:40.027672 (+ 14us) server_negotiation.cc:407] Connection header received
0923 15:05:40.027717 (+ 45us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:05:40.027718 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:05:40.027739 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:05:40.027825 (+ 86us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 15:05:40.027968 (+ 143us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:05:40.028463 (+ 495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:05:40.029238 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:05:40.029452 (+ 214us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:05:40.029539 (+ 87us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:05:40.029695 (+ 156us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 15:05:40.029793 (+ 98us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 15:05:40.029905 (+ 112us) server_negotiation.cc:1035] Waiting for connection context
0923 15:05:40.029936 (+ 31us) server_negotiation.cc:299] Negotiation successful
0923 15:05:40.029948 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260923 15:05:40.032068 31971 raft_consensus.cc:1261] T 2994cb9a8f2d473aa26ca15513cb2ad3 P 4bbff0ac238d4eb3ba3ccc0c6392d82d [term 1 FOLLOWER]: Refusing update from remote peer cb7001eb72124596a2779dcad5e977c4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 15:05:40.032068 32105 raft_consensus.cc:1261] T 2994cb9a8f2d473aa26ca15513cb2ad3 P 1c51cd4531f2488992d5568d0129709c [term 1 FOLLOWER]: Refusing update from remote peer cb7001eb72124596a2779dcad5e977c4: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 15:05:40.032485 32176 consensus_queue.cc:1035] T 2994cb9a8f2d473aa26ca15513cb2ad3 P cb7001eb72124596a2779dcad5e977c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "4bbff0ac238d4eb3ba3ccc0c6392d82d" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 41113 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 15:05:40.032614 32178 consensus_queue.cc:1035] T 2994cb9a8f2d473aa26ca15513cb2ad3 P cb7001eb72124596a2779dcad5e977c4 [LEADER]: Connected to new peer: Peer: permanent_uuid: "1c51cd4531f2488992d5568d0129709c" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 34795 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260923 15:05:40.034229 31884 tablet.cc:2369] T 2994cb9a8f2d473aa26ca15513cb2ad3 P cb7001eb72124596a2779dcad5e977c4: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 15:05:40.035784 32183 mvcc.cc:205] Tried to move back new op lower bound from 7332560650368507904 to 7332560650304446464. Current Snapshot: MvccSnapshot[applied={T|T < 7332560650368507904 or (T in {7332560650368507904})}]
I20260923 15:05:40.036765 32182 mvcc.cc:205] Tried to move back new op lower bound from 7332560650368507904 to 7332560650304446464. Current Snapshot: MvccSnapshot[applied={T|T < 7332560650368507904 or (T in {7332560650368507904})}]
I20260923 15:05:40.037688 31690 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43238:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 15:05:40.037837 31690 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:43238:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 15:05:40.038722 31690 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 9bcdf9a29ef74caea466640f41ae92a1: Sending DeleteTablet for 3 replicas of tablet 2994cb9a8f2d473aa26ca15513cb2ad3
I20260923 15:05:40.039207 31951 tablet_service.cc:1514] Processing DeleteTablet for tablet 2994cb9a8f2d473aa26ca15513cb2ad3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 15:05:40 UTC) from {username='kudu', principal='kudu/127.13.75.190@KRBTEST.COM'} at 127.0.0.1:38476
I20260923 15:05:40.039261 31816 tablet_service.cc:1514] Processing DeleteTablet for tablet 2994cb9a8f2d473aa26ca15513cb2ad3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 15:05:40 UTC) from {username='kudu', principal='kudu/127.13.75.190@KRBTEST.COM'} at 127.0.0.1:46190
I20260923 15:05:40.039548 32191 tablet_replica.cc:330] T 2994cb9a8f2d473aa26ca15513cb2ad3 P cb7001eb72124596a2779dcad5e977c4: stopping tablet replica
I20260923 15:05:40.039705 32191 raft_consensus.cc:2229] T 2994cb9a8f2d473aa26ca15513cb2ad3 P cb7001eb72124596a2779dcad5e977c4 [term 1 LEADER]: Raft consensus shutting down.
I20260923 15:05:40.039777 32085 tablet_service.cc:1514] Processing DeleteTablet for tablet 2994cb9a8f2d473aa26ca15513cb2ad3 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 15:05:40 UTC) from {username='kudu', principal='kudu/127.13.75.190@KRBTEST.COM'} at 127.0.0.1:42212
I20260923 15:05:40.039839 32191 raft_consensus.cc:2258] T 2994cb9a8f2d473aa26ca15513cb2ad3 P cb7001eb72124596a2779dcad5e977c4 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 15:05:40.040184 13614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKC4soy/build/release/bin/kudu with pid 31753
I20260923 15:05:40.043078 32192 tablet_replica.cc:330] T 2994cb9a8f2d473aa26ca15513cb2ad3 P 1c51cd4531f2488992d5568d0129709c: stopping tablet replica
I20260923 15:05:40.043282 32192 raft_consensus.cc:2229] T 2994cb9a8f2d473aa26ca15513cb2ad3 P 1c51cd4531f2488992d5568d0129709c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 15:05:40.043426 32192 raft_consensus.cc:2258] T 2994cb9a8f2d473aa26ca15513cb2ad3 P 1c51cd4531f2488992d5568d0129709c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 15:05:40.043612 32190 tablet_replica.cc:330] T 2994cb9a8f2d473aa26ca15513cb2ad3 P 4bbff0ac238d4eb3ba3ccc0c6392d82d: stopping tablet replica
I20260923 15:05:40.043748 32190 raft_consensus.cc:2229] T 2994cb9a8f2d473aa26ca15513cb2ad3 P 4bbff0ac238d4eb3ba3ccc0c6392d82d [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 15:05:40.043879 32190 raft_consensus.cc:2258] T 2994cb9a8f2d473aa26ca15513cb2ad3 P 4bbff0ac238d4eb3ba3ccc0c6392d82d [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 15:05:40.044040 32192 ts_tablet_manager.cc:1905] T 2994cb9a8f2d473aa26ca15513cb2ad3 P 1c51cd4531f2488992d5568d0129709c: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 15:05:40.044353 32190 ts_tablet_manager.cc:1905] T 2994cb9a8f2d473aa26ca15513cb2ad3 P 4bbff0ac238d4eb3ba3ccc0c6392d82d: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 15:05:40.046000 32192 ts_tablet_manager.cc:1918] T 2994cb9a8f2d473aa26ca15513cb2ad3 P 1c51cd4531f2488992d5568d0129709c: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 15:05:40.046094 32192 log.cc:1196] T 2994cb9a8f2d473aa26ca15513cb2ad3 P 1c51cd4531f2488992d5568d0129709c: Deleting WAL directory at /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175691491353-13614-0/minicluster-data/ts-2/wal/wals/2994cb9a8f2d473aa26ca15513cb2ad3
I20260923 15:05:40.046171 32190 ts_tablet_manager.cc:1918] T 2994cb9a8f2d473aa26ca15513cb2ad3 P 4bbff0ac238d4eb3ba3ccc0c6392d82d: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 15:05:40.046248 32190 log.cc:1196] T 2994cb9a8f2d473aa26ca15513cb2ad3 P 4bbff0ac238d4eb3ba3ccc0c6392d82d: Deleting WAL directory at /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_2.1790175691491353-13614-0/minicluster-data/ts-1/wal/wals/2994cb9a8f2d473aa26ca15513cb2ad3
I20260923 15:05:40.046407 32192 ts_tablet_manager.cc:1939] T 2994cb9a8f2d473aa26ca15513cb2ad3 P 1c51cd4531f2488992d5568d0129709c: Deleting consensus metadata
W20260923 15:05:40.046541 31675 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.75.129:36275 (error 108)
I20260923 15:05:40.046723 32190 ts_tablet_manager.cc:1939] T 2994cb9a8f2d473aa26ca15513cb2ad3 P 4bbff0ac238d4eb3ba3ccc0c6392d82d: Deleting consensus metadata
I20260923 15:05:40.046934 31677 catalog_manager.cc:4841] TS 1c51cd4531f2488992d5568d0129709c (127.13.75.131:34795): tablet 2994cb9a8f2d473aa26ca15513cb2ad3 (table test-table [id=99bc1758cbe7470495b29cd64d2000ba]) successfully deleted
I20260923 15:05:40.047137 31675 catalog_manager.cc:4841] TS 4bbff0ac238d4eb3ba3ccc0c6392d82d (127.13.75.130:41113): tablet 2994cb9a8f2d473aa26ca15513cb2ad3 (table test-table [id=99bc1758cbe7470495b29cd64d2000ba]) successfully deleted
I20260923 15:05:40.047156 32162 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:40.047020 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.129:36275 (local address 127.0.0.1:46222)
0923 15:05:40.047059 (+ 39us) negotiation.cc:107] Waiting for socket to connect
0923 15:05:40.047096 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.129:36275: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":19}
W20260923 15:05:40.047322 31675 catalog_manager.cc:4584] TS cb7001eb72124596a2779dcad5e977c4 (127.13.75.129:36275): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 2994cb9a8f2d473aa26ca15513cb2ad3: Network error: Client connection negotiation failed: client connection to 127.13.75.129:36275: connect: Connection refused (error 111)
I20260923 15:05:40.047351 13614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKC4soy/build/release/bin/kudu with pid 31888
I20260923 15:05:40.052382 13614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKC4soy/build/release/bin/kudu with pid 32022
I20260923 15:05:40.057152 13614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKC4soy/build/release/bin/kudu with pid 31659
2026-09-23T15:05:40Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/2 (2052 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/3
2026-09-23T15:05:40Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T15:05:40Z Disabled control of system clock
I20260923 15:05:40.086022 13614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKC4soy/build/release/bin/kudu
/tmp/dist-test-taskKC4soy/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790175691491353-13614-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790175691491353-13614-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790175691491353-13614-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790175691491353-13614-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.75.190:45671
--webserver_interface=127.13.75.190
--webserver_port=0
--builtin_ntp_servers=127.13.75.148:33203
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.75.190:45671
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260923 15:05:40.168603 32197 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:05:40.168838 32197 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:05:40.168860 32197 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:05:40.169986 32197 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 15:05:40.170019 32197 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:05:40.170032 32197 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:05:40.170044 32197 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 15:05:40.170056 32197 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 15:05:40.171450 32197 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.75.148:33203
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790175691491353-13614-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790175691491353-13614-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.75.190:45671
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.75.190:45671
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790175691491353-13614-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.75.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32197
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790175691491353-13614-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:05:40.171650 32197 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:05:40.171892 32197 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:05:40.176928 32205 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:05:40.177067 32197 server_base.cc:1034] running on GCE node
W20260923 15:05:40.176872 32202 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:05:40.176874 32203 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:05:40.177522 32197 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:05:40.177800 32197 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:05:40.178972 32197 hybrid_clock.cc:648] HybridClock initialized: now 1790175940178955 us; error 30 us; skew 500 ppm
I20260923 15:05:40.184813 32197 webserver.cc:460] Webserver started at http://127.13.75.190:35821/ using document root <none> and password file <none>
I20260923 15:05:40.185128 32197 fs_manager.cc:359] Metadata directory not provided
I20260923 15:05:40.185184 32197 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:05:40.185353 32197 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:05:40.186517 32197 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790175691491353-13614-0/minicluster-data/master-0/data/instance:
uuid: "abd726a144a44b9489876eb0b4b93773"
format_stamp: "Formatted at 2026-09-23 15:05:40 on dist-test-slave-cnb0"
server_key: "f4775c693ec491bca80b6ec85b2cd0d5"
server_key_iv: "9be0d22424831a94c71f52b88991dbfd"
server_key_version: "encryptionkey@0"
I20260923 15:05:40.186940 32197 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790175691491353-13614-0/minicluster-data/master-0/wal/instance:
uuid: "abd726a144a44b9489876eb0b4b93773"
format_stamp: "Formatted at 2026-09-23 15:05:40 on dist-test-slave-cnb0"
server_key: "f4775c693ec491bca80b6ec85b2cd0d5"
server_key_iv: "9be0d22424831a94c71f52b88991dbfd"
server_key_version: "encryptionkey@0"
I20260923 15:05:40.188421 32197 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260923 15:05:40.189324 32211 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:05:40.189533 32197 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260923 15:05:40.189643 32197 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790175691491353-13614-0/minicluster-data/master-0/data,/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790175691491353-13614-0/minicluster-data/master-0/wal
uuid: "abd726a144a44b9489876eb0b4b93773"
format_stamp: "Formatted at 2026-09-23 15:05:40 on dist-test-slave-cnb0"
server_key: "f4775c693ec491bca80b6ec85b2cd0d5"
server_key_iv: "9be0d22424831a94c71f52b88991dbfd"
server_key_version: "encryptionkey@0"
I20260923 15:05:40.189720 32197 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790175691491353-13614-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790175691491353-13614-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790175691491353-13614-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:05:40.203995 32197 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:05:40.204350 32197 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:05:40.204507 32197 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:05:40.208920 32197 rpc_server.cc:307] RPC server started. Bound to: 127.13.75.190:45671
I20260923 15:05:40.208961 32263 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.75.190:45671 every 8 connection(s)
I20260923 15:05:40.209321 32197 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790175691491353-13614-0/minicluster-data/master-0/data/info.pb
I20260923 15:05:40.210351 32264 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:05:40.211081 13614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKC4soy/build/release/bin/kudu as pid 32197
I20260923 15:05:40.211195 13614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790175691491353-13614-0/minicluster-data/master-0/wal/instance
I20260923 15:05:40.211385 13614 external_mini_cluster.cc:1428] Setting key de5d764314eebb96822144e27106faff
I20260923 15:05:40.213452 32264 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P abd726a144a44b9489876eb0b4b93773: Bootstrap starting.
I20260923 15:05:40.214102 32264 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P abd726a144a44b9489876eb0b4b93773: Neither blocks nor log segments found. Creating new log.
I20260923 15:05:40.214432 32264 log.cc:824] T 00000000000000000000000000000000 P abd726a144a44b9489876eb0b4b93773: Log is configured to *not* fsync() on all Append() calls
I20260923 15:05:40.215227 32264 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P abd726a144a44b9489876eb0b4b93773: No bootstrap required, opened a new log
I20260923 15:05:40.217025 32267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:40.211932 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58266 (local address 127.13.75.190:45671)
0923 15:05:40.212121 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0923 15:05:40.212129 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0923 15:05:40.212154 (+ 25us) server_negotiation.cc:407] Connection header received
0923 15:05:40.212602 (+ 448us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:05:40.212608 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:05:40.212670 (+ 62us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:05:40.212789 (+ 119us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:05:40.213031 (+ 242us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:05:40.213968 (+ 937us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:05:40.215030 (+ 1062us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:05:40.215493 (+ 463us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:05:40.215587 (+ 94us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:05:40.216343 (+ 756us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:05:40.216347 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:05:40.216347 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 15:05:40.216521 (+ 174us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:05:40.216558 (+ 37us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 15:05:40.216569 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:05:40.216624 (+ 55us) server_negotiation.cc:1035] Waiting for connection context
0923 15:05:40.216724 (+ 100us) server_negotiation.cc:299] Negotiation successful
0923 15:05:40.216820 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":111,"thread_start_us":83,"threads_started":1}
I20260923 15:05:40.217533 32264 raft_consensus.cc:348] T 00000000000000000000000000000000 P abd726a144a44b9489876eb0b4b93773 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "abd726a144a44b9489876eb0b4b93773" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 45671 } }
I20260923 15:05:40.217674 32264 raft_consensus.cc:374] T 00000000000000000000000000000000 P abd726a144a44b9489876eb0b4b93773 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:05:40.217731 32264 raft_consensus.cc:729] T 00000000000000000000000000000000 P abd726a144a44b9489876eb0b4b93773 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: abd726a144a44b9489876eb0b4b93773, State: Initialized, Role: FOLLOWER
I20260923 15:05:40.217854 32264 consensus_queue.cc:260] T 00000000000000000000000000000000 P abd726a144a44b9489876eb0b4b93773 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "abd726a144a44b9489876eb0b4b93773" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 45671 } }
I20260923 15:05:40.217918 32264 raft_consensus.cc:388] T 00000000000000000000000000000000 P abd726a144a44b9489876eb0b4b93773 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 15:05:40.217939 32264 raft_consensus.cc:482] T 00000000000000000000000000000000 P abd726a144a44b9489876eb0b4b93773 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 15:05:40.217967 32264 raft_consensus.cc:3037] T 00000000000000000000000000000000 P abd726a144a44b9489876eb0b4b93773 [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:05:40.218500 32264 raft_consensus.cc:504] T 00000000000000000000000000000000 P abd726a144a44b9489876eb0b4b93773 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "abd726a144a44b9489876eb0b4b93773" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 45671 } }
I20260923 15:05:40.218586 32264 leader_election.cc:302] T 00000000000000000000000000000000 P abd726a144a44b9489876eb0b4b93773 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: abd726a144a44b9489876eb0b4b93773; no voters:
I20260923 15:05:40.218719 32264 leader_election.cc:288] T 00000000000000000000000000000000 P abd726a144a44b9489876eb0b4b93773 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 15:05:40.218801 32269 raft_consensus.cc:2781] T 00000000000000000000000000000000 P abd726a144a44b9489876eb0b4b93773 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 15:05:40.218986 32269 raft_consensus.cc:686] T 00000000000000000000000000000000 P abd726a144a44b9489876eb0b4b93773 [term 1 LEADER]: Becoming Leader. State: Replica: abd726a144a44b9489876eb0b4b93773, State: Running, Role: LEADER
I20260923 15:05:40.219022 32264 sys_catalog.cc:564] T 00000000000000000000000000000000 P abd726a144a44b9489876eb0b4b93773 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 15:05:40.219084 32269 consensus_queue.cc:237] T 00000000000000000000000000000000 P abd726a144a44b9489876eb0b4b93773 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "abd726a144a44b9489876eb0b4b93773" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 45671 } }
I20260923 15:05:40.219432 32269 sys_catalog.cc:455] T 00000000000000000000000000000000 P abd726a144a44b9489876eb0b4b93773 [sys.catalog]: SysCatalogTable state changed. Reason: New leader abd726a144a44b9489876eb0b4b93773. Latest consensus state: current_term: 1 leader_uuid: "abd726a144a44b9489876eb0b4b93773" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "abd726a144a44b9489876eb0b4b93773" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 45671 } } }
I20260923 15:05:40.219527 32269 sys_catalog.cc:458] T 00000000000000000000000000000000 P abd726a144a44b9489876eb0b4b93773 [sys.catalog]: This master's current role is: LEADER
I20260923 15:05:40.219444 32270 sys_catalog.cc:455] T 00000000000000000000000000000000 P abd726a144a44b9489876eb0b4b93773 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "abd726a144a44b9489876eb0b4b93773" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "abd726a144a44b9489876eb0b4b93773" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 45671 } } }
I20260923 15:05:40.219692 32270 sys_catalog.cc:458] T 00000000000000000000000000000000 P abd726a144a44b9489876eb0b4b93773 [sys.catalog]: This master's current role is: LEADER
W20260923 15:05:40.220829 32285 catalog_manager.cc:1558] T 00000000000000000000000000000000 P abd726a144a44b9489876eb0b4b93773: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260923 15:05:40.220875 32285 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260923 15:05:40.220935 32275 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 15:05:40.221077 32275 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 15:05:40.222811 32275 catalog_manager.cc:1347] Generated new cluster ID: 661d4f3a461b4fa29d618bae3df1b893
I20260923 15:05:40.222870 32275 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 15:05:40.236896 32275 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 15:05:40.237519 32275 catalog_manager.cc:1504] Loading token signing keys...
I20260923 15:05:40.243420 32275 catalog_manager.cc:5868] T 00000000000000000000000000000000 P abd726a144a44b9489876eb0b4b93773: Generated new TSK 0
I20260923 15:05:40.243642 32275 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 15:05:40.249703 13614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKC4soy/build/release/bin/kudu
/tmp/dist-test-taskKC4soy/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790175691491353-13614-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790175691491353-13614-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790175691491353-13614-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790175691491353-13614-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.75.129:0
--local_ip_for_outbound_sockets=127.13.75.129
--webserver_interface=127.13.75.129
--webserver_port=0
--tserver_master_addrs=127.13.75.190:45671
--builtin_ntp_servers=127.13.75.148:33203
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260923 15:05:40.334630 32288 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:05:40.334844 32288 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:05:40.334868 32288 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:05:40.336092 32288 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:05:40.336133 32288 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:05:40.336159 32288 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.75.129
I20260923 15:05:40.337532 32288 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.75.148:33203
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790175691491353-13614-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790175691491353-13614-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.75.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790175691491353-13614-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.75.129
--webserver_port=0
--tserver_master_addrs=127.13.75.190:45671
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32288
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.75.129
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790175691491353-13614-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:05:40.337778 32288 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:05:40.338099 32288 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:05:40.343204 32294 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:05:40.343223 32293 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:05:40.343374 32296 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:05:40.343897 32288 server_base.cc:1034] running on GCE node
I20260923 15:05:40.344117 32288 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:05:40.344400 32288 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:05:40.345575 32288 hybrid_clock.cc:648] HybridClock initialized: now 1790175940345561 us; error 33 us; skew 500 ppm
I20260923 15:05:40.346889 32288 webserver.cc:460] Webserver started at http://127.13.75.129:37957/ using document root <none> and password file <none>
I20260923 15:05:40.347157 32288 fs_manager.cc:359] Metadata directory not provided
I20260923 15:05:40.347226 32288 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:05:40.347448 32288 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:05:40.348573 32288 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790175691491353-13614-0/minicluster-data/ts-0/data/instance:
uuid: "ea0a818c4daf48c6b519cf2063f674ae"
format_stamp: "Formatted at 2026-09-23 15:05:40 on dist-test-slave-cnb0"
server_key: "d87ae2478db68b324bca63856c6a053d"
server_key_iv: "1dfeafeb912f175c77dd7239e37df3ca"
server_key_version: "encryptionkey@0"
I20260923 15:05:40.348940 32288 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790175691491353-13614-0/minicluster-data/ts-0/wal/instance:
uuid: "ea0a818c4daf48c6b519cf2063f674ae"
format_stamp: "Formatted at 2026-09-23 15:05:40 on dist-test-slave-cnb0"
server_key: "d87ae2478db68b324bca63856c6a053d"
server_key_iv: "1dfeafeb912f175c77dd7239e37df3ca"
server_key_version: "encryptionkey@0"
I20260923 15:05:40.350333 32288 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260923 15:05:40.351071 32302 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:05:40.351262 32288 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260923 15:05:40.351356 32288 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790175691491353-13614-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790175691491353-13614-0/minicluster-data/ts-0/wal
uuid: "ea0a818c4daf48c6b519cf2063f674ae"
format_stamp: "Formatted at 2026-09-23 15:05:40 on dist-test-slave-cnb0"
server_key: "d87ae2478db68b324bca63856c6a053d"
server_key_iv: "1dfeafeb912f175c77dd7239e37df3ca"
server_key_version: "encryptionkey@0"
I20260923 15:05:40.351435 32288 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790175691491353-13614-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790175691491353-13614-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790175691491353-13614-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:05:40.372124 32288 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:05:40.372465 32288 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:05:40.372618 32288 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:05:40.372866 32288 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 15:05:40.373286 32288 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 15:05:40.373348 32288 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 15:05:40.373392 32288 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 15:05:40.373420 32288 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 15:05:40.379309 32288 rpc_server.cc:307] RPC server started. Bound to: 127.13.75.129:35157
I20260923 15:05:40.379400 32415 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.75.129:35157 every 8 connection(s)
I20260923 15:05:40.379732 32288 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790175691491353-13614-0/minicluster-data/ts-0/data/info.pb
I20260923 15:05:40.385193 32267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:40.380573 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.129:51315 (local address 127.13.75.190:45671)
0923 15:05:40.380645 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0923 15:05:40.380647 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:05:40.380889 (+ 242us) server_negotiation.cc:407] Connection header received
0923 15:05:40.381374 (+ 485us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:05:40.381375 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:05:40.381389 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:05:40.381493 (+ 104us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:05:40.382176 (+ 683us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:05:40.382687 (+ 511us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:05:40.383708 (+ 1021us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:05:40.383906 (+ 198us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:05:40.383967 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:05:40.384817 (+ 850us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:05:40.384818 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:05:40.384819 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 15:05:40.384823 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:05:40.384838 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 15:05:40.384843 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:05:40.384890 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0923 15:05:40.385092 (+ 202us) server_negotiation.cc:299] Negotiation successful
0923 15:05:40.385131 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260923 15:05:40.385330 13614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKC4soy/build/release/bin/kudu as pid 32288
I20260923 15:05:40.385449 13614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790175691491353-13614-0/minicluster-data/ts-0/wal/instance
I20260923 15:05:40.385644 32418 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:40.380537 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:45671 (local address 127.13.75.129:51315)
0923 15:05:40.380763 (+ 226us) negotiation.cc:107] Waiting for socket to connect
0923 15:05:40.380811 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0923 15:05:40.381295 (+ 484us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:05:40.381572 (+ 277us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:05:40.381577 (+ 5us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:05:40.381585 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:05:40.382100 (+ 515us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:05:40.382105 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:05:40.382785 (+ 680us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:05:40.382787 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:05:40.383626 (+ 839us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:05:40.383628 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:05:40.384013 (+ 385us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:05:40.384014 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:05:40.384167 (+ 153us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:05:40.384669 (+ 502us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 15:05:40.384669 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:05:40.384707 (+ 38us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 15:05:40.384732 (+ 25us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:05:40.384935 (+ 203us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:05:40.384943 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:05:40.384946 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 15:05:40.385080 (+ 134us) client_negotiation.cc:240] Negotiation successful
0923 15:05:40.385218 (+ 138us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":126,"thread_start_us":91,"threads_started":1}
I20260923 15:05:40.385722 13614 external_mini_cluster.cc:1428] Setting key f250c86da79ca11861e049af46402f17
I20260923 15:05:40.386292 32416 heartbeater.cc:347] Connected to a master server at 127.13.75.190:45671
I20260923 15:05:40.386427 32416 heartbeater.cc:464] Registering TS with master...
I20260923 15:05:40.386650 32416 heartbeater.cc:511] Master 127.13.75.190:45671 requested a full tablet report, sending...
I20260923 15:05:40.386984 13614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKC4soy/build/release/bin/kudu
/tmp/dist-test-taskKC4soy/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790175691491353-13614-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790175691491353-13614-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790175691491353-13614-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790175691491353-13614-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.75.130:0
--local_ip_for_outbound_sockets=127.13.75.130
--webserver_interface=127.13.75.130
--webserver_port=0
--tserver_master_addrs=127.13.75.190:45671
--builtin_ntp_servers=127.13.75.148:33203
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260923 15:05:40.387346 32228 ts_manager.cc:194] Registered new tserver with Master: ea0a818c4daf48c6b519cf2063f674ae (127.13.75.129:35157)
I20260923 15:05:40.388106 32228 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.13.75.129:51315
W20260923 15:05:40.470902 32419 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:05:40.471108 32419 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:05:40.471130 32419 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:05:40.472256 32419 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:05:40.472290 32419 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:05:40.472316 32419 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.75.130
I20260923 15:05:40.473567 32419 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.75.148:33203
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790175691491353-13614-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790175691491353-13614-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.75.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790175691491353-13614-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.75.130
--webserver_port=0
--tserver_master_addrs=127.13.75.190:45671
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32419
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.75.130
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790175691491353-13614-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:05:40.473752 32419 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:05:40.473994 32419 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:05:40.479089 32427 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:05:40.479228 32419 server_base.cc:1034] running on GCE node
W20260923 15:05:40.479110 32425 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:05:40.479089 32424 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:05:40.479637 32419 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:05:40.479898 32419 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:05:40.481112 32419 hybrid_clock.cc:648] HybridClock initialized: now 1790175940481104 us; error 35 us; skew 500 ppm
I20260923 15:05:40.482511 32419 webserver.cc:460] Webserver started at http://127.13.75.130:38849/ using document root <none> and password file <none>
I20260923 15:05:40.482775 32419 fs_manager.cc:359] Metadata directory not provided
I20260923 15:05:40.482851 32419 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:05:40.483055 32419 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:05:40.484149 32419 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790175691491353-13614-0/minicluster-data/ts-1/data/instance:
uuid: "db3904c2e9154d5db76cf2596879af4b"
format_stamp: "Formatted at 2026-09-23 15:05:40 on dist-test-slave-cnb0"
server_key: "150fb56f479ea49bd5c45b72129e650c"
server_key_iv: "5a7611038f9e204d903333ce7d37e179"
server_key_version: "encryptionkey@0"
I20260923 15:05:40.484520 32419 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790175691491353-13614-0/minicluster-data/ts-1/wal/instance:
uuid: "db3904c2e9154d5db76cf2596879af4b"
format_stamp: "Formatted at 2026-09-23 15:05:40 on dist-test-slave-cnb0"
server_key: "150fb56f479ea49bd5c45b72129e650c"
server_key_iv: "5a7611038f9e204d903333ce7d37e179"
server_key_version: "encryptionkey@0"
I20260923 15:05:40.485939 32419 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 15:05:40.486733 32433 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:05:40.486954 32419 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:05:40.487053 32419 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790175691491353-13614-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790175691491353-13614-0/minicluster-data/ts-1/wal
uuid: "db3904c2e9154d5db76cf2596879af4b"
format_stamp: "Formatted at 2026-09-23 15:05:40 on dist-test-slave-cnb0"
server_key: "150fb56f479ea49bd5c45b72129e650c"
server_key_iv: "5a7611038f9e204d903333ce7d37e179"
server_key_version: "encryptionkey@0"
I20260923 15:05:40.487139 32419 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790175691491353-13614-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790175691491353-13614-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790175691491353-13614-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:05:40.496281 32419 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:05:40.496637 32419 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:05:40.496801 32419 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:05:40.497051 32419 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 15:05:40.497395 32419 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 15:05:40.497449 32419 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 15:05:40.497490 32419 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 15:05:40.497536 32419 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 15:05:40.503381 32419 rpc_server.cc:307] RPC server started. Bound to: 127.13.75.130:33213
I20260923 15:05:40.503463 32547 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.75.130:33213 every 8 connection(s)
I20260923 15:05:40.503813 32419 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790175691491353-13614-0/minicluster-data/ts-1/data/info.pb
I20260923 15:05:40.509253 32267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:40.504611 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.130:47521 (local address 127.13.75.190:45671)
0923 15:05:40.504675 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0923 15:05:40.504677 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:05:40.504929 (+ 252us) server_negotiation.cc:407] Connection header received
0923 15:05:40.505497 (+ 568us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:05:40.505498 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:05:40.505510 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:05:40.505567 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:05:40.506268 (+ 701us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:05:40.506784 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:05:40.507817 (+ 1033us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:05:40.508018 (+ 201us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:05:40.508104 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:05:40.508947 (+ 843us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:05:40.508948 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:05:40.508949 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 15:05:40.508954 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:05:40.508969 (+ 15us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 15:05:40.508973 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:05:40.509040 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0923 15:05:40.509136 (+ 96us) server_negotiation.cc:299] Negotiation successful
0923 15:05:40.509169 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260923 15:05:40.509531 32550 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:40.504602 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:45671 (local address 127.13.75.130:47521)
0923 15:05:40.504809 (+ 207us) negotiation.cc:107] Waiting for socket to connect
0923 15:05:40.504842 (+ 33us) client_negotiation.cc:174] Beginning negotiation
0923 15:05:40.505385 (+ 543us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:05:40.505645 (+ 260us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:05:40.505648 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:05:40.505657 (+ 9us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:05:40.506169 (+ 512us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:05:40.506173 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:05:40.506922 (+ 749us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:05:40.506923 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:05:40.507708 (+ 785us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:05:40.507710 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:05:40.508117 (+ 407us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:05:40.508118 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:05:40.508251 (+ 133us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:05:40.508795 (+ 544us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 15:05:40.508795 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:05:40.508824 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 15:05:40.508848 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:05:40.509042 (+ 194us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:05:40.509048 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:05:40.509051 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 15:05:40.509137 (+ 86us) client_negotiation.cc:240] Negotiation successful
0923 15:05:40.509240 (+ 103us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":145,"thread_start_us":93,"threads_started":1}
I20260923 15:05:40.510089 32548 heartbeater.cc:347] Connected to a master server at 127.13.75.190:45671
I20260923 15:05:40.510170 32548 heartbeater.cc:464] Registering TS with master...
I20260923 15:05:40.510365 32548 heartbeater.cc:511] Master 127.13.75.190:45671 requested a full tablet report, sending...
I20260923 15:05:40.510779 32228 ts_manager.cc:194] Registered new tserver with Master: db3904c2e9154d5db76cf2596879af4b (127.13.75.130:33213)
I20260923 15:05:40.511258 32228 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.13.75.130:47521
I20260923 15:05:40.512372 13614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKC4soy/build/release/bin/kudu as pid 32419
I20260923 15:05:40.512490 13614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790175691491353-13614-0/minicluster-data/ts-1/wal/instance
I20260923 15:05:40.512708 13614 external_mini_cluster.cc:1428] Setting key 3f259f456db48eb1ffee715838b44f26
I20260923 15:05:40.513820 13614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKC4soy/build/release/bin/kudu
/tmp/dist-test-taskKC4soy/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790175691491353-13614-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790175691491353-13614-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790175691491353-13614-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790175691491353-13614-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.75.131:0
--local_ip_for_outbound_sockets=127.13.75.131
--webserver_interface=127.13.75.131
--webserver_port=0
--tserver_master_addrs=127.13.75.190:45671
--builtin_ntp_servers=127.13.75.148:33203
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260923 15:05:40.597878 32551 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:05:40.598116 32551 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:05:40.598148 32551 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:05:40.599459 32551 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:05:40.599502 32551 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:05:40.599530 32551 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.75.131
I20260923 15:05:40.600924 32551 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.75.148:33203
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790175691491353-13614-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790175691491353-13614-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.75.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790175691491353-13614-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.75.131
--webserver_port=0
--tserver_master_addrs=127.13.75.190:45671
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32551
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.75.131
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790175691491353-13614-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:05:40.601200 32551 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:05:40.601537 32551 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:05:40.606721 32557 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:05:40.606729 32556 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:05:40.606792 32559 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:05:40.607097 32551 server_base.cc:1034] running on GCE node
I20260923 15:05:40.607432 32551 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:05:40.607720 32551 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:05:40.608917 32551 hybrid_clock.cc:648] HybridClock initialized: now 1790175940608911 us; error 41 us; skew 500 ppm
I20260923 15:05:40.610256 32551 webserver.cc:460] Webserver started at http://127.13.75.131:44245/ using document root <none> and password file <none>
I20260923 15:05:40.610522 32551 fs_manager.cc:359] Metadata directory not provided
I20260923 15:05:40.610598 32551 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:05:40.610752 32551 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:05:40.612000 32551 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790175691491353-13614-0/minicluster-data/ts-2/data/instance:
uuid: "a9063eb7e712427ebcfd6b74ede66615"
format_stamp: "Formatted at 2026-09-23 15:05:40 on dist-test-slave-cnb0"
server_key: "f940231543f283bf961df3e6a046bed2"
server_key_iv: "c722ff37cf17c3f4ba28bb6f11596df3"
server_key_version: "encryptionkey@0"
I20260923 15:05:40.612385 32551 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790175691491353-13614-0/minicluster-data/ts-2/wal/instance:
uuid: "a9063eb7e712427ebcfd6b74ede66615"
format_stamp: "Formatted at 2026-09-23 15:05:40 on dist-test-slave-cnb0"
server_key: "f940231543f283bf961df3e6a046bed2"
server_key_iv: "c722ff37cf17c3f4ba28bb6f11596df3"
server_key_version: "encryptionkey@0"
I20260923 15:05:40.613759 32551 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260923 15:05:40.614624 32565 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:05:40.614830 32551 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260923 15:05:40.614976 32551 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790175691491353-13614-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790175691491353-13614-0/minicluster-data/ts-2/wal
uuid: "a9063eb7e712427ebcfd6b74ede66615"
format_stamp: "Formatted at 2026-09-23 15:05:40 on dist-test-slave-cnb0"
server_key: "f940231543f283bf961df3e6a046bed2"
server_key_iv: "c722ff37cf17c3f4ba28bb6f11596df3"
server_key_version: "encryptionkey@0"
I20260923 15:05:40.615064 32551 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790175691491353-13614-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790175691491353-13614-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790175691491353-13614-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:05:40.640872 32551 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:05:40.641234 32551 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:05:40.641398 32551 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:05:40.641662 32551 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 15:05:40.642026 32551 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 15:05:40.642083 32551 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 15:05:40.642119 32551 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 15:05:40.642162 32551 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 15:05:40.648036 32551 rpc_server.cc:307] RPC server started. Bound to: 127.13.75.131:41683
I20260923 15:05:40.648123 32678 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.75.131:41683 every 8 connection(s)
I20260923 15:05:40.648476 32551 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790175691491353-13614-0/minicluster-data/ts-2/data/info.pb
I20260923 15:05:40.649076 13614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKC4soy/build/release/bin/kudu as pid 32551
I20260923 15:05:40.649185 13614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790175691491353-13614-0/minicluster-data/ts-2/wal/instance
I20260923 15:05:40.649422 13614 external_mini_cluster.cc:1428] Setting key d36a093f69d8a995bc37d9cc8a6c94f8
I20260923 15:05:40.654153 32267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:40.649276 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.131:55415 (local address 127.13.75.190:45671)
0923 15:05:40.649347 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0923 15:05:40.649350 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:05:40.649687 (+ 337us) server_negotiation.cc:407] Connection header received
0923 15:05:40.650306 (+ 619us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:05:40.650307 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:05:40.650318 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:05:40.650384 (+ 66us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:05:40.651207 (+ 823us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:05:40.651704 (+ 497us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:05:40.652752 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:05:40.652945 (+ 193us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:05:40.653002 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:05:40.653924 (+ 922us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:05:40.653925 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:05:40.653925 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 15:05:40.653930 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:05:40.653944 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 15:05:40.653947 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:05:40.653991 (+ 44us) server_negotiation.cc:1035] Waiting for connection context
0923 15:05:40.654066 (+ 75us) server_negotiation.cc:299] Negotiation successful
0923 15:05:40.654089 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260923 15:05:40.654371 32681 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:40.649340 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:45671 (local address 127.13.75.131:55415)
0923 15:05:40.649539 (+ 199us) negotiation.cc:107] Waiting for socket to connect
0923 15:05:40.649582 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0923 15:05:40.650170 (+ 588us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:05:40.650478 (+ 308us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:05:40.650482 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:05:40.650507 (+ 25us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:05:40.651097 (+ 590us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:05:40.651103 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:05:40.651829 (+ 726us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:05:40.651830 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:05:40.652649 (+ 819us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:05:40.652651 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:05:40.653021 (+ 370us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:05:40.653021 (+ 0us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:05:40.653139 (+ 118us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:05:40.653787 (+ 648us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 15:05:40.653787 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:05:40.653819 (+ 32us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 15:05:40.653843 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:05:40.653991 (+ 148us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:05:40.653994 (+ 3us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:05:40.653996 (+ 2us) client_negotiation.cc:769] Sending connection context
0923 15:05:40.654044 (+ 48us) client_negotiation.cc:240] Negotiation successful
0923 15:05:40.654126 (+ 82us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":102,"threads_started":1}
I20260923 15:05:40.654917 32679 heartbeater.cc:347] Connected to a master server at 127.13.75.190:45671
I20260923 15:05:40.655014 32679 heartbeater.cc:464] Registering TS with master...
I20260923 15:05:40.655174 32679 heartbeater.cc:511] Master 127.13.75.190:45671 requested a full tablet report, sending...
I20260923 15:05:40.655570 32228 ts_manager.cc:194] Registered new tserver with Master: a9063eb7e712427ebcfd6b74ede66615 (127.13.75.131:41683)
I20260923 15:05:40.655912 32228 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.13.75.131:55415
I20260923 15:05:40.661624 13614 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260923 15:05:40.666278 32267 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:40.663202 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:58280 (local address 127.13.75.190:45671)
0923 15:05:40.663255 (+ 53us) server_negotiation.cc:206] Beginning negotiation
0923 15:05:40.663262 (+ 7us) server_negotiation.cc:399] Waiting for connection header
0923 15:05:40.663298 (+ 36us) server_negotiation.cc:407] Connection header received
0923 15:05:40.663354 (+ 56us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:05:40.663355 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:05:40.663363 (+ 8us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:05:40.663416 (+ 53us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:05:40.663550 (+ 134us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:05:40.664087 (+ 537us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:05:40.664927 (+ 840us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:05:40.665134 (+ 207us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:05:40.665219 (+ 85us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:05:40.665955 (+ 736us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:05:40.665957 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:05:40.665957 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 15:05:40.665962 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:05:40.665984 (+ 22us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 15:05:40.665987 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:05:40.666089 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0923 15:05:40.666173 (+ 84us) server_negotiation.cc:299] Negotiation successful
0923 15:05:40.666185 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260923 15:05:40.668172 32228 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:58280:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 15:05:40.669955 32228 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 15:05:40.676867 32691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:40.672705 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33846 (local address 127.13.75.130:33213)
0923 15:05:40.672900 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0923 15:05:40.672905 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 15:05:40.673001 (+ 96us) server_negotiation.cc:407] Connection header received
0923 15:05:40.673052 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:05:40.673054 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:05:40.673070 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:05:40.673161 (+ 91us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:05:40.673399 (+ 238us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:05:40.674611 (+ 1212us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:05:40.675904 (+ 1293us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:05:40.676424 (+ 520us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:05:40.676526 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:05:40.676629 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0923 15:05:40.676718 (+ 89us) server_negotiation.cc:299] Negotiation successful
0923 15:05:40.676760 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":129,"thread_start_us":86,"threads_started":1}
I20260923 15:05:40.676882 32689 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:40.672448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.130:33213 (local address 127.0.0.1:33846)
0923 15:05:40.672930 (+ 482us) negotiation.cc:107] Waiting for socket to connect
0923 15:05:40.672939 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 15:05:40.672990 (+ 51us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:05:40.673248 (+ 258us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:05:40.673252 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:05:40.673254 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:05:40.673331 (+ 77us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:05:40.673333 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:05:40.674690 (+ 1357us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:05:40.674691 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:05:40.675822 (+ 1131us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:05:40.675824 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:05:40.676530 (+ 706us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:05:40.676531 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:05:40.676605 (+ 74us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:05:40.676626 (+ 21us) client_negotiation.cc:769] Sending connection context
0923 15:05:40.676703 (+ 77us) client_negotiation.cc:240] Negotiation successful
0923 15:05:40.676760 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":466,"mutex_wait_us":35,"thread_start_us":109,"threads_started":1}
I20260923 15:05:40.676894 32687 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:40.672447 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.131:41683 (local address 127.0.0.1:41200)
0923 15:05:40.672565 (+ 118us) negotiation.cc:107] Waiting for socket to connect
0923 15:05:40.672608 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0923 15:05:40.672684 (+ 76us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:05:40.673293 (+ 609us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:05:40.673293 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:05:40.673295 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:05:40.673455 (+ 160us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:05:40.673457 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:05:40.674374 (+ 917us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:05:40.674375 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:05:40.675678 (+ 1303us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:05:40.675681 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:05:40.676637 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:05:40.676638 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:05:40.676718 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:05:40.676722 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 15:05:40.676753 (+ 31us) client_negotiation.cc:240] Negotiation successful
0923 15:05:40.676794 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":93,"thread_start_us":70,"threads_started":1}
I20260923 15:05:40.677071 32688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:40.672448 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.129:35157 (local address 127.0.0.1:57632)
0923 15:05:40.672900 (+ 452us) negotiation.cc:107] Waiting for socket to connect
0923 15:05:40.672912 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 15:05:40.672969 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:05:40.673340 (+ 371us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:05:40.673341 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:05:40.673344 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:05:40.673393 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:05:40.673395 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:05:40.674702 (+ 1307us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:05:40.674703 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:05:40.675856 (+ 1153us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:05:40.675860 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:05:40.676839 (+ 979us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:05:40.676840 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:05:40.676895 (+ 55us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:05:40.676898 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 15:05:40.676943 (+ 45us) client_negotiation.cc:240] Negotiation successful
0923 15:05:40.676985 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":432,"mutex_wait_us":2,"thread_start_us":110,"threads_started":1}
I20260923 15:05:40.677388 32692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:40.672744 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57632 (local address 127.13.75.129:35157)
0923 15:05:40.672938 (+ 194us) server_negotiation.cc:206] Beginning negotiation
0923 15:05:40.672942 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 15:05:40.672964 (+ 22us) server_negotiation.cc:407] Connection header received
0923 15:05:40.673051 (+ 87us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:05:40.673053 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:05:40.673070 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:05:40.673159 (+ 89us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:05:40.673479 (+ 320us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:05:40.674598 (+ 1119us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:05:40.675956 (+ 1358us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:05:40.676654 (+ 698us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:05:40.676715 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:05:40.676806 (+ 91us) server_negotiation.cc:1035] Waiting for connection context
0923 15:05:40.677265 (+ 459us) server_negotiation.cc:299] Negotiation successful
0923 15:05:40.677307 (+ 42us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":101,"threads_started":1}
I20260923 15:05:40.677476 32690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:40.672629 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41200 (local address 127.13.75.131:41683)
0923 15:05:40.673137 (+ 508us) server_negotiation.cc:206] Beginning negotiation
0923 15:05:40.673141 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 15:05:40.673152 (+ 11us) server_negotiation.cc:407] Connection header received
0923 15:05:40.673195 (+ 43us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:05:40.673197 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:05:40.673213 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:05:40.673286 (+ 73us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:05:40.673536 (+ 250us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:05:40.674258 (+ 722us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:05:40.675769 (+ 1511us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:05:40.676530 (+ 761us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:05:40.677076 (+ 546us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:05:40.677319 (+ 243us) server_negotiation.cc:1035] Waiting for connection context
0923 15:05:40.677342 (+ 23us) server_negotiation.cc:299] Negotiation successful
0923 15:05:40.677387 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":442,"thread_start_us":86,"threads_started":1}
I20260923 15:05:40.677942 32482 tablet_service.cc:1467] Processing CreateTablet for tablet 98456eb47ddf48629768f2e53911f035 (DEFAULT_TABLE table=test-table [id=a18ece9a63424ef1a10b23229e25dd49]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 15:05:40.677971 32350 tablet_service.cc:1467] Processing CreateTablet for tablet 98456eb47ddf48629768f2e53911f035 (DEFAULT_TABLE table=test-table [id=a18ece9a63424ef1a10b23229e25dd49]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 15:05:40.678303 32482 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 98456eb47ddf48629768f2e53911f035. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:05:40.678313 32350 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 98456eb47ddf48629768f2e53911f035. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:05:40.679610 32613 tablet_service.cc:1467] Processing CreateTablet for tablet 98456eb47ddf48629768f2e53911f035 (DEFAULT_TABLE table=test-table [id=a18ece9a63424ef1a10b23229e25dd49]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 15:05:40.679955 32613 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 98456eb47ddf48629768f2e53911f035. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:05:40.681113 32693 tablet_bootstrap.cc:492] T 98456eb47ddf48629768f2e53911f035 P db3904c2e9154d5db76cf2596879af4b: Bootstrap starting.
I20260923 15:05:40.681526 32694 tablet_bootstrap.cc:492] T 98456eb47ddf48629768f2e53911f035 P ea0a818c4daf48c6b519cf2063f674ae: Bootstrap starting.
I20260923 15:05:40.681929 32693 tablet_bootstrap.cc:654] T 98456eb47ddf48629768f2e53911f035 P db3904c2e9154d5db76cf2596879af4b: Neither blocks nor log segments found. Creating new log.
I20260923 15:05:40.682175 32694 tablet_bootstrap.cc:654] T 98456eb47ddf48629768f2e53911f035 P ea0a818c4daf48c6b519cf2063f674ae: Neither blocks nor log segments found. Creating new log.
I20260923 15:05:40.682335 32693 log.cc:824] T 98456eb47ddf48629768f2e53911f035 P db3904c2e9154d5db76cf2596879af4b: Log is configured to *not* fsync() on all Append() calls
I20260923 15:05:40.682483 32694 log.cc:824] T 98456eb47ddf48629768f2e53911f035 P ea0a818c4daf48c6b519cf2063f674ae: Log is configured to *not* fsync() on all Append() calls
I20260923 15:05:40.683158 32693 tablet_bootstrap.cc:492] T 98456eb47ddf48629768f2e53911f035 P db3904c2e9154d5db76cf2596879af4b: No bootstrap required, opened a new log
I20260923 15:05:40.683233 32694 tablet_bootstrap.cc:492] T 98456eb47ddf48629768f2e53911f035 P ea0a818c4daf48c6b519cf2063f674ae: No bootstrap required, opened a new log
I20260923 15:05:40.683234 32693 ts_tablet_manager.cc:1397] T 98456eb47ddf48629768f2e53911f035 P db3904c2e9154d5db76cf2596879af4b: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 15:05:40.683291 32694 ts_tablet_manager.cc:1397] T 98456eb47ddf48629768f2e53911f035 P ea0a818c4daf48c6b519cf2063f674ae: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 15:05:40.683490 32697 tablet_bootstrap.cc:492] T 98456eb47ddf48629768f2e53911f035 P a9063eb7e712427ebcfd6b74ede66615: Bootstrap starting.
I20260923 15:05:40.684051 32697 tablet_bootstrap.cc:654] T 98456eb47ddf48629768f2e53911f035 P a9063eb7e712427ebcfd6b74ede66615: Neither blocks nor log segments found. Creating new log.
I20260923 15:05:40.684336 32697 log.cc:824] T 98456eb47ddf48629768f2e53911f035 P a9063eb7e712427ebcfd6b74ede66615: Log is configured to *not* fsync() on all Append() calls
I20260923 15:05:40.685120 32697 tablet_bootstrap.cc:492] T 98456eb47ddf48629768f2e53911f035 P a9063eb7e712427ebcfd6b74ede66615: No bootstrap required, opened a new log
I20260923 15:05:40.685210 32697 ts_tablet_manager.cc:1397] T 98456eb47ddf48629768f2e53911f035 P a9063eb7e712427ebcfd6b74ede66615: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 15:05:40.685804 32694 raft_consensus.cc:348] T 98456eb47ddf48629768f2e53911f035 P ea0a818c4daf48c6b519cf2063f674ae [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db3904c2e9154d5db76cf2596879af4b" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 33213 } } peers { permanent_uuid: "a9063eb7e712427ebcfd6b74ede66615" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 41683 } } peers { permanent_uuid: "ea0a818c4daf48c6b519cf2063f674ae" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 35157 } }
I20260923 15:05:40.685806 32693 raft_consensus.cc:348] T 98456eb47ddf48629768f2e53911f035 P db3904c2e9154d5db76cf2596879af4b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db3904c2e9154d5db76cf2596879af4b" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 33213 } } peers { permanent_uuid: "a9063eb7e712427ebcfd6b74ede66615" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 41683 } } peers { permanent_uuid: "ea0a818c4daf48c6b519cf2063f674ae" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 35157 } }
I20260923 15:05:40.685976 32693 raft_consensus.cc:374] T 98456eb47ddf48629768f2e53911f035 P db3904c2e9154d5db76cf2596879af4b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:05:40.685976 32694 raft_consensus.cc:374] T 98456eb47ddf48629768f2e53911f035 P ea0a818c4daf48c6b519cf2063f674ae [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:05:40.686012 32694 raft_consensus.cc:729] T 98456eb47ddf48629768f2e53911f035 P ea0a818c4daf48c6b519cf2063f674ae [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ea0a818c4daf48c6b519cf2063f674ae, State: Initialized, Role: FOLLOWER
I20260923 15:05:40.686012 32693 raft_consensus.cc:729] T 98456eb47ddf48629768f2e53911f035 P db3904c2e9154d5db76cf2596879af4b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: db3904c2e9154d5db76cf2596879af4b, State: Initialized, Role: FOLLOWER
I20260923 15:05:40.686137 32694 consensus_queue.cc:260] T 98456eb47ddf48629768f2e53911f035 P ea0a818c4daf48c6b519cf2063f674ae [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db3904c2e9154d5db76cf2596879af4b" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 33213 } } peers { permanent_uuid: "a9063eb7e712427ebcfd6b74ede66615" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 41683 } } peers { permanent_uuid: "ea0a818c4daf48c6b519cf2063f674ae" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 35157 } }
I20260923 15:05:40.686146 32693 consensus_queue.cc:260] T 98456eb47ddf48629768f2e53911f035 P db3904c2e9154d5db76cf2596879af4b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db3904c2e9154d5db76cf2596879af4b" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 33213 } } peers { permanent_uuid: "a9063eb7e712427ebcfd6b74ede66615" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 41683 } } peers { permanent_uuid: "ea0a818c4daf48c6b519cf2063f674ae" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 35157 } }
I20260923 15:05:40.686409 32693 ts_tablet_manager.cc:1428] T 98456eb47ddf48629768f2e53911f035 P db3904c2e9154d5db76cf2596879af4b: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260923 15:05:40.686409 32694 ts_tablet_manager.cc:1428] T 98456eb47ddf48629768f2e53911f035 P ea0a818c4daf48c6b519cf2063f674ae: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260923 15:05:40.686460 32548 heartbeater.cc:503] Master 127.13.75.190:45671 was elected leader, sending a full tablet report...
I20260923 15:05:40.686774 32416 heartbeater.cc:503] Master 127.13.75.190:45671 was elected leader, sending a full tablet report...
I20260923 15:05:40.687606 32697 raft_consensus.cc:348] T 98456eb47ddf48629768f2e53911f035 P a9063eb7e712427ebcfd6b74ede66615 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db3904c2e9154d5db76cf2596879af4b" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 33213 } } peers { permanent_uuid: "a9063eb7e712427ebcfd6b74ede66615" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 41683 } } peers { permanent_uuid: "ea0a818c4daf48c6b519cf2063f674ae" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 35157 } }
I20260923 15:05:40.687765 32697 raft_consensus.cc:374] T 98456eb47ddf48629768f2e53911f035 P a9063eb7e712427ebcfd6b74ede66615 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:05:40.687825 32697 raft_consensus.cc:729] T 98456eb47ddf48629768f2e53911f035 P a9063eb7e712427ebcfd6b74ede66615 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a9063eb7e712427ebcfd6b74ede66615, State: Initialized, Role: FOLLOWER
I20260923 15:05:40.687934 32697 consensus_queue.cc:260] T 98456eb47ddf48629768f2e53911f035 P a9063eb7e712427ebcfd6b74ede66615 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db3904c2e9154d5db76cf2596879af4b" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 33213 } } peers { permanent_uuid: "a9063eb7e712427ebcfd6b74ede66615" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 41683 } } peers { permanent_uuid: "ea0a818c4daf48c6b519cf2063f674ae" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 35157 } }
I20260923 15:05:40.688249 32697 ts_tablet_manager.cc:1428] T 98456eb47ddf48629768f2e53911f035 P a9063eb7e712427ebcfd6b74ede66615: Time spent starting tablet: real 0.003s user 0.000s sys 0.003s
I20260923 15:05:40.688298 32679 heartbeater.cc:503] Master 127.13.75.190:45671 was elected leader, sending a full tablet report...
W20260923 15:05:40.754383 32549 tablet.cc:2369] T 98456eb47ddf48629768f2e53911f035 P db3904c2e9154d5db76cf2596879af4b: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260923 15:05:40.880488 32417 tablet.cc:2369] T 98456eb47ddf48629768f2e53911f035 P ea0a818c4daf48c6b519cf2063f674ae: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260923 15:05:40.899070 32680 tablet.cc:2369] T 98456eb47ddf48629768f2e53911f035 P a9063eb7e712427ebcfd6b74ede66615: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 15:05:41.013363 32700 raft_consensus.cc:482] T 98456eb47ddf48629768f2e53911f035 P ea0a818c4daf48c6b519cf2063f674ae [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 15:05:41.013506 32700 raft_consensus.cc:504] T 98456eb47ddf48629768f2e53911f035 P ea0a818c4daf48c6b519cf2063f674ae [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db3904c2e9154d5db76cf2596879af4b" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 33213 } } peers { permanent_uuid: "a9063eb7e712427ebcfd6b74ede66615" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 41683 } } peers { permanent_uuid: "ea0a818c4daf48c6b519cf2063f674ae" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 35157 } }
I20260923 15:05:41.014133 32700 leader_election.cc:288] T 98456eb47ddf48629768f2e53911f035 P ea0a818c4daf48c6b519cf2063f674ae [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers db3904c2e9154d5db76cf2596879af4b (127.13.75.130:33213), a9063eb7e712427ebcfd6b74ede66615 (127.13.75.131:41683)
I20260923 15:05:41.017187 32690 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:41.014483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.129:39291 (local address 127.13.75.131:41683)
0923 15:05:41.014549 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0923 15:05:41.014552 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:05:41.014611 (+ 59us) server_negotiation.cc:407] Connection header received
0923 15:05:41.014691 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:05:41.014692 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:05:41.014713 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:05:41.014781 (+ 68us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:05:41.014978 (+ 197us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:05:41.015457 (+ 479us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:05:41.016452 (+ 995us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:05:41.016829 (+ 377us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:05:41.016912 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:05:41.016921 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0923 15:05:41.017074 (+ 153us) server_negotiation.cc:299] Negotiation successful
0923 15:05:41.017114 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":36}
I20260923 15:05:41.017210 32703 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:41.014416 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.131:41683 (local address 127.13.75.129:39291)
0923 15:05:41.014548 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0923 15:05:41.014560 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 15:05:41.014628 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:05:41.014787 (+ 159us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:05:41.014788 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:05:41.014790 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:05:41.014869 (+ 79us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:05:41.014873 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:05:41.015555 (+ 682us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:05:41.015556 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:05:41.016354 (+ 798us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:05:41.016357 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:05:41.016912 (+ 555us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:05:41.016913 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:05:41.016999 (+ 86us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:05:41.017003 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 15:05:41.017070 (+ 67us) client_negotiation.cc:240] Negotiation successful
0923 15:05:41.017110 (+ 40us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":117,"thread_start_us":72,"threads_started":1}
I20260923 15:05:41.017539 32691 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:41.014483 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.129:40407 (local address 127.13.75.130:33213)
0923 15:05:41.014567 (+ 84us) server_negotiation.cc:206] Beginning negotiation
0923 15:05:41.014570 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:05:41.014663 (+ 93us) server_negotiation.cc:407] Connection header received
0923 15:05:41.014768 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:05:41.014769 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:05:41.014781 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:05:41.014858 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:05:41.015058 (+ 200us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:05:41.015581 (+ 523us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:05:41.016675 (+ 1094us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:05:41.017059 (+ 384us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:05:41.017168 (+ 109us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:05:41.017177 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0923 15:05:41.017393 (+ 216us) server_negotiation.cc:299] Negotiation successful
0923 15:05:41.017439 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":55}
I20260923 15:05:41.017587 32633 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "98456eb47ddf48629768f2e53911f035" candidate_uuid: "ea0a818c4daf48c6b519cf2063f674ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9063eb7e712427ebcfd6b74ede66615" is_pre_election: true
I20260923 15:05:41.017755 32633 raft_consensus.cc:2454] T 98456eb47ddf48629768f2e53911f035 P a9063eb7e712427ebcfd6b74ede66615 [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ea0a818c4daf48c6b519cf2063f674ae in term 0.
I20260923 15:05:41.017782 32702 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:41.014419 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.130:33213 (local address 127.13.75.129:40407)
0923 15:05:41.014562 (+ 143us) negotiation.cc:107] Waiting for socket to connect
0923 15:05:41.014575 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0923 15:05:41.014681 (+ 106us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:05:41.014876 (+ 195us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:05:41.014877 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:05:41.014879 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:05:41.014982 (+ 103us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:05:41.014984 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:05:41.015711 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:05:41.015712 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:05:41.016548 (+ 836us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:05:41.016550 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:05:41.017172 (+ 622us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:05:41.017173 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:05:41.017285 (+ 112us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:05:41.017290 (+ 5us) client_negotiation.cc:769] Sending connection context
0923 15:05:41.017653 (+ 363us) client_negotiation.cc:240] Negotiation successful
0923 15:05:41.017700 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":125,"thread_start_us":64,"threads_started":1}
I20260923 15:05:41.018051 32304 leader_election.cc:302] T 98456eb47ddf48629768f2e53911f035 P ea0a818c4daf48c6b519cf2063f674ae [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: a9063eb7e712427ebcfd6b74ede66615, ea0a818c4daf48c6b519cf2063f674ae; no voters:
I20260923 15:05:41.018097 32502 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "98456eb47ddf48629768f2e53911f035" candidate_uuid: "ea0a818c4daf48c6b519cf2063f674ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db3904c2e9154d5db76cf2596879af4b" is_pre_election: true
I20260923 15:05:41.018251 32502 raft_consensus.cc:2454] T 98456eb47ddf48629768f2e53911f035 P db3904c2e9154d5db76cf2596879af4b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate ea0a818c4daf48c6b519cf2063f674ae in term 0.
I20260923 15:05:41.018245 32700 raft_consensus.cc:2781] T 98456eb47ddf48629768f2e53911f035 P ea0a818c4daf48c6b519cf2063f674ae [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 15:05:41.018318 32700 raft_consensus.cc:482] T 98456eb47ddf48629768f2e53911f035 P ea0a818c4daf48c6b519cf2063f674ae [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 15:05:41.018352 32700 raft_consensus.cc:3037] T 98456eb47ddf48629768f2e53911f035 P ea0a818c4daf48c6b519cf2063f674ae [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:05:41.019374 32700 raft_consensus.cc:504] T 98456eb47ddf48629768f2e53911f035 P ea0a818c4daf48c6b519cf2063f674ae [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db3904c2e9154d5db76cf2596879af4b" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 33213 } } peers { permanent_uuid: "a9063eb7e712427ebcfd6b74ede66615" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 41683 } } peers { permanent_uuid: "ea0a818c4daf48c6b519cf2063f674ae" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 35157 } }
I20260923 15:05:41.019812 32700 leader_election.cc:288] T 98456eb47ddf48629768f2e53911f035 P ea0a818c4daf48c6b519cf2063f674ae [CANDIDATE]: Term 1 election: Requested vote from peers db3904c2e9154d5db76cf2596879af4b (127.13.75.130:33213), a9063eb7e712427ebcfd6b74ede66615 (127.13.75.131:41683)
I20260923 15:05:41.019827 32502 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "98456eb47ddf48629768f2e53911f035" candidate_uuid: "ea0a818c4daf48c6b519cf2063f674ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "db3904c2e9154d5db76cf2596879af4b"
I20260923 15:05:41.019933 32502 raft_consensus.cc:3037] T 98456eb47ddf48629768f2e53911f035 P db3904c2e9154d5db76cf2596879af4b [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:05:41.019984 32633 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "98456eb47ddf48629768f2e53911f035" candidate_uuid: "ea0a818c4daf48c6b519cf2063f674ae" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "a9063eb7e712427ebcfd6b74ede66615"
I20260923 15:05:41.020087 32633 raft_consensus.cc:3037] T 98456eb47ddf48629768f2e53911f035 P a9063eb7e712427ebcfd6b74ede66615 [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:05:41.021040 32502 raft_consensus.cc:2454] T 98456eb47ddf48629768f2e53911f035 P db3904c2e9154d5db76cf2596879af4b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ea0a818c4daf48c6b519cf2063f674ae in term 1.
I20260923 15:05:41.021160 32633 raft_consensus.cc:2454] T 98456eb47ddf48629768f2e53911f035 P a9063eb7e712427ebcfd6b74ede66615 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate ea0a818c4daf48c6b519cf2063f674ae in term 1.
I20260923 15:05:41.021288 32305 leader_election.cc:302] T 98456eb47ddf48629768f2e53911f035 P ea0a818c4daf48c6b519cf2063f674ae [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: db3904c2e9154d5db76cf2596879af4b, ea0a818c4daf48c6b519cf2063f674ae; no voters:
I20260923 15:05:41.021387 32700 raft_consensus.cc:2781] T 98456eb47ddf48629768f2e53911f035 P ea0a818c4daf48c6b519cf2063f674ae [term 1 FOLLOWER]: Leader election won for term 1
I20260923 15:05:41.021584 32700 raft_consensus.cc:686] T 98456eb47ddf48629768f2e53911f035 P ea0a818c4daf48c6b519cf2063f674ae [term 1 LEADER]: Becoming Leader. State: Replica: ea0a818c4daf48c6b519cf2063f674ae, State: Running, Role: LEADER
I20260923 15:05:41.021698 32700 consensus_queue.cc:237] T 98456eb47ddf48629768f2e53911f035 P ea0a818c4daf48c6b519cf2063f674ae [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db3904c2e9154d5db76cf2596879af4b" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 33213 } } peers { permanent_uuid: "a9063eb7e712427ebcfd6b74ede66615" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 41683 } } peers { permanent_uuid: "ea0a818c4daf48c6b519cf2063f674ae" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 35157 } }
I20260923 15:05:41.022586 32227 catalog_manager.cc:5495] T 98456eb47ddf48629768f2e53911f035 P ea0a818c4daf48c6b519cf2063f674ae reported cstate change: term changed from 0 to 1, leader changed from <none> to ea0a818c4daf48c6b519cf2063f674ae (127.13.75.129). New cstate: current_term: 1 leader_uuid: "ea0a818c4daf48c6b519cf2063f674ae" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "db3904c2e9154d5db76cf2596879af4b" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 33213 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "a9063eb7e712427ebcfd6b74ede66615" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 41683 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "ea0a818c4daf48c6b519cf2063f674ae" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 35157 } health_report { overall_health: HEALTHY } } }
I20260923 15:05:41.035701 32692 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:41.032972 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:57646 (local address 127.13.75.129:35157)
0923 15:05:41.033036 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0923 15:05:41.033037 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 15:05:41.033046 (+ 9us) server_negotiation.cc:407] Connection header received
0923 15:05:41.033172 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:05:41.033173 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:05:41.033199 (+ 26us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:05:41.033275 (+ 76us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 15:05:41.033445 (+ 170us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:05:41.033967 (+ 522us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:05:41.034780 (+ 813us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:05:41.035005 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:05:41.035070 (+ 65us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:05:41.035320 (+ 250us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 15:05:41.035410 (+ 90us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 15:05:41.035488 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0923 15:05:41.035541 (+ 53us) server_negotiation.cc:299] Negotiation successful
0923 15:05:41.035551 (+ 10us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260923 15:05:41.037739 32502 raft_consensus.cc:1261] T 98456eb47ddf48629768f2e53911f035 P db3904c2e9154d5db76cf2596879af4b [term 1 FOLLOWER]: Refusing update from remote peer ea0a818c4daf48c6b519cf2063f674ae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 15:05:41.037739 32633 raft_consensus.cc:1261] T 98456eb47ddf48629768f2e53911f035 P a9063eb7e712427ebcfd6b74ede66615 [term 1 FOLLOWER]: Refusing update from remote peer ea0a818c4daf48c6b519cf2063f674ae: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 15:05:41.038134 32704 consensus_queue.cc:1035] T 98456eb47ddf48629768f2e53911f035 P ea0a818c4daf48c6b519cf2063f674ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "a9063eb7e712427ebcfd6b74ede66615" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 41683 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 15:05:41.038271 32700 consensus_queue.cc:1035] T 98456eb47ddf48629768f2e53911f035 P ea0a818c4daf48c6b519cf2063f674ae [LEADER]: Connected to new peer: Peer: permanent_uuid: "db3904c2e9154d5db76cf2596879af4b" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 33213 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 15:05:41.043262 32227 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:58280:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 15:05:41.043372 32227 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:58280:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 15:05:41.044128 32227 catalog_manager.cc:5782] T 00000000000000000000000000000000 P abd726a144a44b9489876eb0b4b93773: Sending DeleteTablet for 3 replicas of tablet 98456eb47ddf48629768f2e53911f035
I20260923 15:05:41.044462 32613 tablet_service.cc:1514] Processing DeleteTablet for tablet 98456eb47ddf48629768f2e53911f035 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 15:05:41 UTC) from {username='slave'} at 127.0.0.1:41200
I20260923 15:05:41.044538 32482 tablet_service.cc:1514] Processing DeleteTablet for tablet 98456eb47ddf48629768f2e53911f035 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 15:05:41 UTC) from {username='slave'} at 127.0.0.1:33846
I20260923 15:05:41.044708 32717 tablet_replica.cc:330] T 98456eb47ddf48629768f2e53911f035 P a9063eb7e712427ebcfd6b74ede66615: stopping tablet replica
I20260923 15:05:41.044736 32350 tablet_service.cc:1514] Processing DeleteTablet for tablet 98456eb47ddf48629768f2e53911f035 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 15:05:41 UTC) from {username='slave'} at 127.0.0.1:57632
I20260923 15:05:41.044756 13614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKC4soy/build/release/bin/kudu with pid 32288
I20260923 15:05:41.044813 32717 raft_consensus.cc:2229] T 98456eb47ddf48629768f2e53911f035 P a9063eb7e712427ebcfd6b74ede66615 [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 15:05:41.045727 32718 tablet_replica.cc:330] T 98456eb47ddf48629768f2e53911f035 P db3904c2e9154d5db76cf2596879af4b: stopping tablet replica
I20260923 15:05:41.045852 32718 raft_consensus.cc:2229] T 98456eb47ddf48629768f2e53911f035 P db3904c2e9154d5db76cf2596879af4b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 15:05:41.045977 32718 raft_consensus.cc:2258] T 98456eb47ddf48629768f2e53911f035 P db3904c2e9154d5db76cf2596879af4b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 15:05:41.046838 32718 ts_tablet_manager.cc:1905] T 98456eb47ddf48629768f2e53911f035 P db3904c2e9154d5db76cf2596879af4b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 15:05:41.044927 32717 raft_consensus.cc:2258] T 98456eb47ddf48629768f2e53911f035 P a9063eb7e712427ebcfd6b74ede66615 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 15:05:41.047683 32717 ts_tablet_manager.cc:1905] T 98456eb47ddf48629768f2e53911f035 P a9063eb7e712427ebcfd6b74ede66615: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 15:05:41.048789 32718 ts_tablet_manager.cc:1918] T 98456eb47ddf48629768f2e53911f035 P db3904c2e9154d5db76cf2596879af4b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 15:05:41.048882 32718 log.cc:1196] T 98456eb47ddf48629768f2e53911f035 P db3904c2e9154d5db76cf2596879af4b: Deleting WAL directory at /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790175691491353-13614-0/minicluster-data/ts-1/wal/wals/98456eb47ddf48629768f2e53911f035
I20260923 15:05:41.049203 32718 ts_tablet_manager.cc:1939] T 98456eb47ddf48629768f2e53911f035 P db3904c2e9154d5db76cf2596879af4b: Deleting consensus metadata
I20260923 15:05:41.049722 32214 catalog_manager.cc:4841] TS db3904c2e9154d5db76cf2596879af4b (127.13.75.130:33213): tablet 98456eb47ddf48629768f2e53911f035 (table test-table [id=a18ece9a63424ef1a10b23229e25dd49]) successfully deleted
I20260923 15:05:41.049878 32717 ts_tablet_manager.cc:1918] T 98456eb47ddf48629768f2e53911f035 P a9063eb7e712427ebcfd6b74ede66615: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 15:05:41.049984 32717 log.cc:1196] T 98456eb47ddf48629768f2e53911f035 P a9063eb7e712427ebcfd6b74ede66615: Deleting WAL directory at /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_3.1790175691491353-13614-0/minicluster-data/ts-2/wal/wals/98456eb47ddf48629768f2e53911f035
I20260923 15:05:41.050269 32717 ts_tablet_manager.cc:1939] T 98456eb47ddf48629768f2e53911f035 P a9063eb7e712427ebcfd6b74ede66615: Deleting consensus metadata
W20260923 15:05:41.050468 32215 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.75.129:35157 (error 108)
I20260923 15:05:41.050727 13614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKC4soy/build/release/bin/kudu with pid 32419
I20260923 15:05:41.050788 32213 catalog_manager.cc:4841] TS a9063eb7e712427ebcfd6b74ede66615 (127.13.75.131:41683): tablet 98456eb47ddf48629768f2e53911f035 (table test-table [id=a18ece9a63424ef1a10b23229e25dd49]) successfully deleted
I20260923 15:05:41.051429 32688 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:41.050921 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.129:35157 (local address 127.0.0.1:57658)
0923 15:05:41.051345 (+ 424us) negotiation.cc:107] Waiting for socket to connect
0923 15:05:41.051389 (+ 44us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.129:35157: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":402}
W20260923 15:05:41.051853 32215 catalog_manager.cc:4584] TS ea0a818c4daf48c6b519cf2063f674ae (127.13.75.129:35157): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 98456eb47ddf48629768f2e53911f035: Network error: Client connection negotiation failed: client connection to 127.13.75.129:35157: connect: Connection refused (error 111)
I20260923 15:05:41.056223 13614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKC4soy/build/release/bin/kudu with pid 32551
I20260923 15:05:41.060935 13614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKC4soy/build/release/bin/kudu with pid 32197
2026-09-23T15:05:41Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/3 (1002 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/4
Loading random data
Initializing database '/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175691491353-13614-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 15:05:41 dist-test-slave-cnb0 krb5kdc[32723](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 15:05:41 dist-test-slave-cnb0 krb5kdc[32723](info): set up 2 sockets
Sep 23 15:05:41 dist-test-slave-cnb0 krb5kdc[32723](info): commencing operation
krb5kdc: starting...
W20260923 15:05:42.103906 13614 mini_kdc.cc:121] Time spent starting KDC: real 1.023s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175691491353-13614-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175691491353-13614-0/krb5kdc/test-user.keytab.
Sep 23 15:05:42 dist-test-slave-cnb0 krb5kdc[32723](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175942, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T15:05:42Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T15:05:42Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.75.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.75.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.75.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.190.keytab.
Entry for principal kudu/127.13.75.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.190.keytab.
I20260923 15:05:42.229797 13614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKC4soy/build/release/bin/kudu
/tmp/dist-test-taskKC4soy/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175691491353-13614-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175691491353-13614-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175691491353-13614-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175691491353-13614-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.75.190:35527
--webserver_interface=127.13.75.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.190.keytab
--principal=kudu/127.13.75.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.75.148:42141
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.75.190:35527
--encrypt_data_at_rest=true
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175691491353-13614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175691491353-13614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175691491353-13614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:05:42.313017 32736 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:05:42.313288 32736 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:05:42.313328 32736 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:05:42.314496 32736 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 15:05:42.314566 32736 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:05:42.314592 32736 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:05:42.314623 32736 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 15:05:42.314648 32736 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 15:05:42.316248 32736 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.75.148:42141
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175691491353-13614-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175691491353-13614-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.75.190:35527
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.190.keytab
--principal=kudu/127.13.75.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.75.190:35527
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175691491353-13614-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.75.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.32736
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175691491353-13614-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:05:42.316542 32736 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:05:42.316783 32736 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:05:42.321861 32741 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:05:42.322032 32736 server_base.cc:1034] running on GCE node
W20260923 15:05:42.321861 32745 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:05:42.321907 32742 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:05:42.322469 32736 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:05:42.322784 32736 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:05:42.323935 32736 hybrid_clock.cc:648] HybridClock initialized: now 1790175942323914 us; error 42 us; skew 500 ppm
Sep 23 15:05:42 dist-test-slave-cnb0 krb5kdc[32723](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175942, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:05:42.326364 32736 init.cc:377] Logged in from keytab as kudu/127.13.75.190@KRBTEST.COM (short username kudu)
I20260923 15:05:42.326823 32736 webserver.cc:460] Webserver started at http://127.13.75.190:46541/ using document root <none> and password file <none>
I20260923 15:05:42.327082 32736 fs_manager.cc:359] Metadata directory not provided
I20260923 15:05:42.327129 32736 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:05:42.327225 32736 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:05:42.328227 32736 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175691491353-13614-0/minicluster-data/master-0/data/instance:
uuid: "137f102e9b8b460f94101de0862c072c"
format_stamp: "Formatted at 2026-09-23 15:05:42 on dist-test-slave-cnb0"
server_key: "0d9b5a370d22a7280879d6d7d581b7c3"
server_key_iv: "6120c30ae606ab42f166dc42052243da"
server_key_version: "encryptionkey@0"
I20260923 15:05:42.328523 32736 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175691491353-13614-0/minicluster-data/master-0/wal/instance:
uuid: "137f102e9b8b460f94101de0862c072c"
format_stamp: "Formatted at 2026-09-23 15:05:42 on dist-test-slave-cnb0"
server_key: "0d9b5a370d22a7280879d6d7d581b7c3"
server_key_iv: "6120c30ae606ab42f166dc42052243da"
server_key_version: "encryptionkey@0"
I20260923 15:05:42.329823 32736 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 15:05:42.330509 32752 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:05:42.330789 32736 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:05:42.330946 32736 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175691491353-13614-0/minicluster-data/master-0/data,/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175691491353-13614-0/minicluster-data/master-0/wal
uuid: "137f102e9b8b460f94101de0862c072c"
format_stamp: "Formatted at 2026-09-23 15:05:42 on dist-test-slave-cnb0"
server_key: "0d9b5a370d22a7280879d6d7d581b7c3"
server_key_iv: "6120c30ae606ab42f166dc42052243da"
server_key_version: "encryptionkey@0"
I20260923 15:05:42.331049 32736 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175691491353-13614-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175691491353-13614-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175691491353-13614-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:05:42.350132 32736 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:05:42.352576 32736 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:05:42.352758 32736 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:05:42.357085 32736 rpc_server.cc:307] RPC server started. Bound to: 127.13.75.190:35527
I20260923 15:05:42.357120 336 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.75.190:35527 every 8 connection(s)
I20260923 15:05:42.357494 32736 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175691491353-13614-0/minicluster-data/master-0/data/info.pb
I20260923 15:05:42.358392 337 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:05:42.360774 337 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 137f102e9b8b460f94101de0862c072c: Bootstrap starting.
I20260923 15:05:42.361395 337 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 137f102e9b8b460f94101de0862c072c: Neither blocks nor log segments found. Creating new log.
I20260923 15:05:42.361702 337 log.cc:824] T 00000000000000000000000000000000 P 137f102e9b8b460f94101de0862c072c: Log is configured to *not* fsync() on all Append() calls
I20260923 15:05:42.362506 337 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 137f102e9b8b460f94101de0862c072c: No bootstrap required, opened a new log
I20260923 15:05:42.364262 337 raft_consensus.cc:348] T 00000000000000000000000000000000 P 137f102e9b8b460f94101de0862c072c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "137f102e9b8b460f94101de0862c072c" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 35527 } }
I20260923 15:05:42.364427 337 raft_consensus.cc:374] T 00000000000000000000000000000000 P 137f102e9b8b460f94101de0862c072c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:05:42.364452 337 raft_consensus.cc:729] T 00000000000000000000000000000000 P 137f102e9b8b460f94101de0862c072c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 137f102e9b8b460f94101de0862c072c, State: Initialized, Role: FOLLOWER
I20260923 15:05:42.364614 337 consensus_queue.cc:260] T 00000000000000000000000000000000 P 137f102e9b8b460f94101de0862c072c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "137f102e9b8b460f94101de0862c072c" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 35527 } }
I20260923 15:05:42.364696 337 raft_consensus.cc:388] T 00000000000000000000000000000000 P 137f102e9b8b460f94101de0862c072c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 15:05:42.364748 337 raft_consensus.cc:482] T 00000000000000000000000000000000 P 137f102e9b8b460f94101de0862c072c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 15:05:42.364801 337 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 137f102e9b8b460f94101de0862c072c [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:05:42.365394 13614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKC4soy/build/release/bin/kudu as pid 32736
I20260923 15:05:42.365422 337 raft_consensus.cc:504] T 00000000000000000000000000000000 P 137f102e9b8b460f94101de0862c072c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "137f102e9b8b460f94101de0862c072c" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 35527 } }
I20260923 15:05:42.365643 13614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175691491353-13614-0/minicluster-data/master-0/wal/instance
I20260923 15:05:42.365624 337 leader_election.cc:302] T 00000000000000000000000000000000 P 137f102e9b8b460f94101de0862c072c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 137f102e9b8b460f94101de0862c072c; no voters:
I20260923 15:05:42.365855 13614 external_mini_cluster.cc:1428] Setting key 27b1701d27088d022253fcfdffab9de9
I20260923 15:05:42.365854 337 leader_election.cc:288] T 00000000000000000000000000000000 P 137f102e9b8b460f94101de0862c072c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 15:05:42.365890 340 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 137f102e9b8b460f94101de0862c072c [term 1 FOLLOWER]: Leader election won for term 1
I20260923 15:05:42.366014 340 raft_consensus.cc:686] T 00000000000000000000000000000000 P 137f102e9b8b460f94101de0862c072c [term 1 LEADER]: Becoming Leader. State: Replica: 137f102e9b8b460f94101de0862c072c, State: Running, Role: LEADER
I20260923 15:05:42.366123 340 consensus_queue.cc:237] T 00000000000000000000000000000000 P 137f102e9b8b460f94101de0862c072c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "137f102e9b8b460f94101de0862c072c" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 35527 } }
I20260923 15:05:42.366153 337 sys_catalog.cc:564] T 00000000000000000000000000000000 P 137f102e9b8b460f94101de0862c072c [sys.catalog]: configured and running, proceeding with master startup.
I20260923 15:05:42.366508 340 sys_catalog.cc:455] T 00000000000000000000000000000000 P 137f102e9b8b460f94101de0862c072c [sys.catalog]: SysCatalogTable state changed. Reason: New leader 137f102e9b8b460f94101de0862c072c. Latest consensus state: current_term: 1 leader_uuid: "137f102e9b8b460f94101de0862c072c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "137f102e9b8b460f94101de0862c072c" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 35527 } } }
I20260923 15:05:42.366578 340 sys_catalog.cc:458] T 00000000000000000000000000000000 P 137f102e9b8b460f94101de0862c072c [sys.catalog]: This master's current role is: LEADER
I20260923 15:05:42.366956 341 sys_catalog.cc:455] T 00000000000000000000000000000000 P 137f102e9b8b460f94101de0862c072c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "137f102e9b8b460f94101de0862c072c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "137f102e9b8b460f94101de0862c072c" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 35527 } } }
I20260923 15:05:42.367178 341 sys_catalog.cc:458] T 00000000000000000000000000000000 P 137f102e9b8b460f94101de0862c072c [sys.catalog]: This master's current role is: LEADER
I20260923 15:05:42.367331 345 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 15:05:42.368162 345 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 15:05:42.370848 345 catalog_manager.cc:1347] Generated new cluster ID: f078ccd3c6be47d38cfba54d53080951
I20260923 15:05:42.370937 345 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 23 15:05:42 dist-test-slave-cnb0 krb5kdc[32723](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175942, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.75.190@KRBTEST.COM
I20260923 15:05:42.380122 346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:42.366780 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44204 (local address 127.13.75.190:35527)
0923 15:05:42.367380 (+ 600us) server_negotiation.cc:206] Beginning negotiation
0923 15:05:42.367441 (+ 61us) server_negotiation.cc:399] Waiting for connection header
0923 15:05:42.367485 (+ 44us) server_negotiation.cc:407] Connection header received
0923 15:05:42.367636 (+ 151us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:05:42.367644 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:05:42.367753 (+ 109us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:05:42.368823 (+ 1070us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:05:42.368935 (+ 112us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:05:42.370275 (+ 1340us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:05:42.371291 (+ 1016us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:05:42.371987 (+ 696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:05:42.372114 (+ 127us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:05:42.375125 (+ 3011us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:05:42.375131 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:05:42.375131 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:05:42.375147 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:05:42.378152 (+ 3005us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:05:42.378624 (+ 472us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:05:42.378628 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:05:42.378628 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:05:42.378774 (+ 146us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:05:42.379012 (+ 238us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:05:42.379013 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:05:42.379013 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:05:42.379382 (+ 369us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:05:42.379551 (+ 169us) server_negotiation.cc:1035] Waiting for connection context
0923 15:05:42.379694 (+ 143us) server_negotiation.cc:299] Negotiation successful
0923 15:05:42.379860 (+ 166us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":544,"thread_start_us":64,"threads_started":1}
I20260923 15:05:42.390166 345 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 15:05:42.390828 345 catalog_manager.cc:1504] Loading token signing keys...
I20260923 15:05:42.392982 345 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 137f102e9b8b460f94101de0862c072c: Generated new TSK 0
I20260923 15:05:42.393191 345 catalog_manager.cc:1514] Initializing in-progress tserver states...
WARNING: no policy specified for kudu/127.13.75.129@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.75.129@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.75.129 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.129.keytab.
Entry for principal kudu/127.13.75.129 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.129.keytab.
I20260923 15:05:42.423878 13614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKC4soy/build/release/bin/kudu
/tmp/dist-test-taskKC4soy/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175691491353-13614-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175691491353-13614-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175691491353-13614-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175691491353-13614-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.75.129:0
--local_ip_for_outbound_sockets=127.13.75.129
--webserver_interface=127.13.75.129
--webserver_port=0
--tserver_master_addrs=127.13.75.190:35527
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.129.keytab
--principal=kudu/127.13.75.129
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.75.148:42141
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175691491353-13614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175691491353-13614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175691491353-13614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:05:42.510268 363 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:05:42.510478 363 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:05:42.510504 363 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:05:42.511641 363 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:05:42.511683 363 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:05:42.511776 363 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.75.129
I20260923 15:05:42.513281 363 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.75.148:42141
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175691491353-13614-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175691491353-13614-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.129.keytab
--principal=kudu/127.13.75.129
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.75.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175691491353-13614-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.75.129
--webserver_port=0
--tserver_master_addrs=127.13.75.190:35527
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.363
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.75.129
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175691491353-13614-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:05:42.513484 363 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:05:42.513672 363 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:05:42.518834 371 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:05:42.518822 368 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:05:42.519049 363 server_base.cc:1034] running on GCE node
W20260923 15:05:42.518843 369 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:05:42.519394 363 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:05:42.519629 363 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:05:42.520773 363 hybrid_clock.cc:648] HybridClock initialized: now 1790175942520742 us; error 53 us; skew 500 ppm
Sep 23 15:05:42 dist-test-slave-cnb0 krb5kdc[32723](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175942, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.129@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:05:42.523504 363 init.cc:377] Logged in from keytab as kudu/127.13.75.129@KRBTEST.COM (short username kudu)
I20260923 15:05:42.523962 363 webserver.cc:460] Webserver started at http://127.13.75.129:36963/ using document root <none> and password file <none>
I20260923 15:05:42.524252 363 fs_manager.cc:359] Metadata directory not provided
I20260923 15:05:42.524336 363 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:05:42.524492 363 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:05:42.525568 363 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175691491353-13614-0/minicluster-data/ts-0/data/instance:
uuid: "12b7cc036e1d4585ac0afd587eaaeb78"
format_stamp: "Formatted at 2026-09-23 15:05:42 on dist-test-slave-cnb0"
server_key: "700e0fdccc730f47ce7f139b114df122"
server_key_iv: "dac0058d342d2953f20c75a531434192"
server_key_version: "encryptionkey@0"
I20260923 15:05:42.525938 363 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175691491353-13614-0/minicluster-data/ts-0/wal/instance:
uuid: "12b7cc036e1d4585ac0afd587eaaeb78"
format_stamp: "Formatted at 2026-09-23 15:05:42 on dist-test-slave-cnb0"
server_key: "700e0fdccc730f47ce7f139b114df122"
server_key_iv: "dac0058d342d2953f20c75a531434192"
server_key_version: "encryptionkey@0"
I20260923 15:05:42.527383 363 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260923 15:05:42.528129 378 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:05:42.528354 363 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260923 15:05:42.528471 363 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175691491353-13614-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175691491353-13614-0/minicluster-data/ts-0/wal
uuid: "12b7cc036e1d4585ac0afd587eaaeb78"
format_stamp: "Formatted at 2026-09-23 15:05:42 on dist-test-slave-cnb0"
server_key: "700e0fdccc730f47ce7f139b114df122"
server_key_iv: "dac0058d342d2953f20c75a531434192"
server_key_version: "encryptionkey@0"
I20260923 15:05:42.528556 363 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175691491353-13614-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175691491353-13614-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175691491353-13614-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:05:42.539031 363 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:05:42.541510 363 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:05:42.541707 363 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:05:42.541975 363 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 15:05:42.542305 363 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 15:05:42.542362 363 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 15:05:42.542404 363 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 15:05:42.542429 363 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 15:05:42.548799 363 rpc_server.cc:307] RPC server started. Bound to: 127.13.75.129:45255
I20260923 15:05:42.548856 491 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.75.129:45255 every 8 connection(s)
I20260923 15:05:42.549278 363 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175691491353-13614-0/minicluster-data/ts-0/data/info.pb
Sep 23 15:05:42 dist-test-slave-cnb0 krb5kdc[32723](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175942, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.129@KRBTEST.COM for kudu/127.13.75.190@KRBTEST.COM
I20260923 15:05:42.559566 13614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKC4soy/build/release/bin/kudu as pid 363
I20260923 15:05:42.559669 346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:42.550056 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.129:46971 (local address 127.13.75.190:35527)
0923 15:05:42.550127 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0923 15:05:42.550129 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:05:42.550377 (+ 248us) server_negotiation.cc:407] Connection header received
0923 15:05:42.550951 (+ 574us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:05:42.550953 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:05:42.550968 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:05:42.551050 (+ 82us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:05:42.552105 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:05:42.552623 (+ 518us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:05:42.553726 (+ 1103us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:05:42.553966 (+ 240us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:05:42.554044 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:05:42.556562 (+ 2518us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:05:42.556564 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:05:42.556564 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:05:42.556572 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:05:42.558072 (+ 1500us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:05:42.558489 (+ 417us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:05:42.558490 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:05:42.558490 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:05:42.558543 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:05:42.558871 (+ 328us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:05:42.558872 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:05:42.558872 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:05:42.559055 (+ 183us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:05:42.559400 (+ 345us) server_negotiation.cc:1035] Waiting for connection context
0923 15:05:42.559454 (+ 54us) server_negotiation.cc:299] Negotiation successful
0923 15:05:42.559556 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":42}
I20260923 15:05:42.559688 13614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175691491353-13614-0/minicluster-data/ts-0/wal/instance
I20260923 15:05:42.559731 494 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:42.550031 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:35527 (local address 127.13.75.129:46971)
0923 15:05:42.550233 (+ 202us) negotiation.cc:107] Waiting for socket to connect
0923 15:05:42.550281 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0923 15:05:42.550810 (+ 529us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:05:42.551113 (+ 303us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:05:42.551117 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:05:42.551786 (+ 669us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:05:42.552000 (+ 214us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:05:42.552005 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:05:42.552733 (+ 728us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:05:42.552734 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:05:42.553612 (+ 878us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:05:42.553614 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:05:42.554059 (+ 445us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:05:42.554060 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:05:42.554177 (+ 117us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:05:42.554764 (+ 587us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:05:42.554774 (+ 10us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:05:42.556468 (+ 1694us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:05:42.558159 (+ 1691us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:05:42.558168 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:05:42.558171 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:05:42.558413 (+ 242us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:05:42.558658 (+ 245us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:05:42.558659 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:05:42.558660 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:05:42.558782 (+ 122us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:05:42.559137 (+ 355us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:05:42.559143 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:05:42.559273 (+ 130us) client_negotiation.cc:769] Sending connection context
0923 15:05:42.559371 (+ 98us) client_negotiation.cc:240] Negotiation successful
0923 15:05:42.559508 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":150,"thread_start_us":100,"threads_started":1}
I20260923 15:05:42.559899 13614 external_mini_cluster.cc:1428] Setting key 5a2425f6e659256de45539b13b67db08
I20260923 15:05:42.560377 492 heartbeater.cc:347] Connected to a master server at 127.13.75.190:35527
I20260923 15:05:42.560485 492 heartbeater.cc:464] Registering TS with master...
I20260923 15:05:42.560724 492 heartbeater.cc:511] Master 127.13.75.190:35527 requested a full tablet report, sending...
I20260923 15:05:42.561332 301 ts_manager.cc:194] Registered new tserver with Master: 12b7cc036e1d4585ac0afd587eaaeb78 (127.13.75.129:45255)
I20260923 15:05:42.561990 301 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.75.129@KRBTEST.COM'} at 127.13.75.129:46971
WARNING: no policy specified for kudu/127.13.75.130@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.75.130@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.75.130 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.130.keytab.
Entry for principal kudu/127.13.75.130 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.130.keytab.
I20260923 15:05:42.587703 13614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKC4soy/build/release/bin/kudu
/tmp/dist-test-taskKC4soy/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175691491353-13614-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175691491353-13614-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175691491353-13614-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175691491353-13614-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.75.130:0
--local_ip_for_outbound_sockets=127.13.75.130
--webserver_interface=127.13.75.130
--webserver_port=0
--tserver_master_addrs=127.13.75.190:35527
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.130.keytab
--principal=kudu/127.13.75.130
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.75.148:42141
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175691491353-13614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175691491353-13614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175691491353-13614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:05:42.673038 497 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:05:42.673260 497 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:05:42.673390 497 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:05:42.674705 497 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:05:42.674759 497 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:05:42.674788 497 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.75.130
I20260923 15:05:42.676414 497 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.75.148:42141
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175691491353-13614-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175691491353-13614-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.130.keytab
--principal=kudu/127.13.75.130
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.75.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175691491353-13614-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.75.130
--webserver_port=0
--tserver_master_addrs=127.13.75.190:35527
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.497
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.75.130
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175691491353-13614-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:05:42.676698 497 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:05:42.676959 497 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:05:42.682395 503 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:05:42.682415 497 server_base.cc:1034] running on GCE node
W20260923 15:05:42.682463 502 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:05:42.682395 505 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:05:42.682974 497 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:05:42.683234 497 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:05:42.684391 497 hybrid_clock.cc:648] HybridClock initialized: now 1790175942684380 us; error 33 us; skew 500 ppm
Sep 23 15:05:42 dist-test-slave-cnb0 krb5kdc[32723](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175942, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.130@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:05:42.687165 497 init.cc:377] Logged in from keytab as kudu/127.13.75.130@KRBTEST.COM (short username kudu)
I20260923 15:05:42.687650 497 webserver.cc:460] Webserver started at http://127.13.75.130:34901/ using document root <none> and password file <none>
I20260923 15:05:42.687893 497 fs_manager.cc:359] Metadata directory not provided
I20260923 15:05:42.687964 497 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:05:42.688122 497 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:05:42.689224 497 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175691491353-13614-0/minicluster-data/ts-1/data/instance:
uuid: "e2bd2678911741f0bf628d30ec52b3dc"
format_stamp: "Formatted at 2026-09-23 15:05:42 on dist-test-slave-cnb0"
server_key: "4c009df58c7cf8964d586de74eb5f8f5"
server_key_iv: "d86a3cf2bbec79b3a72bdc946eb4eb6c"
server_key_version: "encryptionkey@0"
I20260923 15:05:42.689599 497 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175691491353-13614-0/minicluster-data/ts-1/wal/instance:
uuid: "e2bd2678911741f0bf628d30ec52b3dc"
format_stamp: "Formatted at 2026-09-23 15:05:42 on dist-test-slave-cnb0"
server_key: "4c009df58c7cf8964d586de74eb5f8f5"
server_key_iv: "d86a3cf2bbec79b3a72bdc946eb4eb6c"
server_key_version: "encryptionkey@0"
I20260923 15:05:42.691066 497 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 15:05:42.691941 512 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:05:42.692147 497 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:05:42.692257 497 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175691491353-13614-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175691491353-13614-0/minicluster-data/ts-1/wal
uuid: "e2bd2678911741f0bf628d30ec52b3dc"
format_stamp: "Formatted at 2026-09-23 15:05:42 on dist-test-slave-cnb0"
server_key: "4c009df58c7cf8964d586de74eb5f8f5"
server_key_iv: "d86a3cf2bbec79b3a72bdc946eb4eb6c"
server_key_version: "encryptionkey@0"
I20260923 15:05:42.692334 497 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175691491353-13614-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175691491353-13614-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175691491353-13614-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:05:42.713303 497 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:05:42.715919 497 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:05:42.716092 497 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:05:42.716346 497 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 15:05:42.716686 497 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 15:05:42.716750 497 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 15:05:42.716809 497 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 15:05:42.716840 497 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 15:05:42.722998 497 rpc_server.cc:307] RPC server started. Bound to: 127.13.75.130:44663
I20260923 15:05:42.723069 625 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.75.130:44663 every 8 connection(s)
I20260923 15:05:42.723433 497 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175691491353-13614-0/minicluster-data/ts-1/data/info.pb
Sep 23 15:05:42 dist-test-slave-cnb0 krb5kdc[32723](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175942, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.130@KRBTEST.COM for kudu/127.13.75.190@KRBTEST.COM
I20260923 15:05:42.733372 13614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKC4soy/build/release/bin/kudu as pid 497
I20260923 15:05:42.733512 13614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175691491353-13614-0/minicluster-data/ts-1/wal/instance
I20260923 15:05:42.733544 346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:42.724248 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.130:45393 (local address 127.13.75.190:35527)
0923 15:05:42.724317 (+ 69us) server_negotiation.cc:206] Beginning negotiation
0923 15:05:42.724319 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:05:42.724548 (+ 229us) server_negotiation.cc:407] Connection header received
0923 15:05:42.725090 (+ 542us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:05:42.725092 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:05:42.725106 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:05:42.725157 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:05:42.726145 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:05:42.726652 (+ 507us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:05:42.727762 (+ 1110us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:05:42.727966 (+ 204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:05:42.728056 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:05:42.730315 (+ 2259us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:05:42.730316 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:05:42.730316 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:05:42.730324 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:05:42.731889 (+ 1565us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:05:42.732337 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:05:42.732339 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:05:42.732339 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:05:42.732399 (+ 60us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:05:42.732752 (+ 353us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:05:42.732753 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:05:42.732753 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:05:42.732887 (+ 134us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:05:42.732960 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0923 15:05:42.733281 (+ 321us) server_negotiation.cc:299] Negotiation successful
0923 15:05:42.733418 (+ 137us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260923 15:05:42.733755 13614 external_mini_cluster.cc:1428] Setting key 662ab7dfa656d2bc677247cd649fd2df
I20260923 15:05:42.733781 628 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:42.724243 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:35527 (local address 127.13.75.130:45393)
0923 15:05:42.724417 (+ 174us) negotiation.cc:107] Waiting for socket to connect
0923 15:05:42.724457 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0923 15:05:42.724970 (+ 513us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:05:42.725233 (+ 263us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:05:42.725242 (+ 9us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:05:42.725835 (+ 593us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:05:42.726034 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:05:42.726039 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:05:42.726766 (+ 727us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:05:42.726767 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:05:42.727659 (+ 892us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:05:42.727662 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:05:42.728068 (+ 406us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:05:42.728070 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:05:42.728200 (+ 130us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:05:42.728742 (+ 542us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:05:42.728754 (+ 12us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:05:42.730223 (+ 1469us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:05:42.731984 (+ 1761us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:05:42.731990 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:05:42.731993 (+ 3us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:05:42.732248 (+ 255us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:05:42.732499 (+ 251us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:05:42.732501 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:05:42.732501 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:05:42.732656 (+ 155us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:05:42.732974 (+ 318us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:05:42.732981 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:05:42.733133 (+ 152us) client_negotiation.cc:769] Sending connection context
0923 15:05:42.733259 (+ 126us) client_negotiation.cc:240] Negotiation successful
0923 15:05:42.733441 (+ 182us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":134,"thread_start_us":97,"threads_started":1}
I20260923 15:05:42.734390 626 heartbeater.cc:347] Connected to a master server at 127.13.75.190:35527
I20260923 15:05:42.734537 626 heartbeater.cc:464] Registering TS with master...
I20260923 15:05:42.734850 626 heartbeater.cc:511] Master 127.13.75.190:35527 requested a full tablet report, sending...
I20260923 15:05:42.735474 300 ts_manager.cc:194] Registered new tserver with Master: e2bd2678911741f0bf628d30ec52b3dc (127.13.75.130:44663)
I20260923 15:05:42.736081 300 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.75.130@KRBTEST.COM'} at 127.13.75.130:45393
WARNING: no policy specified for kudu/127.13.75.131@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.75.131@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.75.131 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.131.keytab.
Entry for principal kudu/127.13.75.131 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.131.keytab.
I20260923 15:05:42.763545 13614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKC4soy/build/release/bin/kudu
/tmp/dist-test-taskKC4soy/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175691491353-13614-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175691491353-13614-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175691491353-13614-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175691491353-13614-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.75.131:0
--local_ip_for_outbound_sockets=127.13.75.131
--webserver_interface=127.13.75.131
--webserver_port=0
--tserver_master_addrs=127.13.75.190:35527
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.131.keytab
--principal=kudu/127.13.75.131
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.75.148:42141
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation with env {KRB5CCNAME=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175691491353-13614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175691491353-13614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175691491353-13614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:05:42.848202 631 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:05:42.848429 631 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:05:42.848454 631 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:05:42.849707 631 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:05:42.849767 631 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:05:42.849797 631 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.75.131
I20260923 15:05:42.851513 631 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.75.148:42141
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175691491353-13614-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175691491353-13614-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.131.keytab
--principal=kudu/127.13.75.131
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.75.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175691491353-13614-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.75.131
--webserver_port=0
--tserver_master_addrs=127.13.75.190:35527
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.631
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.75.131
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175691491353-13614-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:05:42.851796 631 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:05:42.852047 631 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:05:42.857357 639 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:05:42.857434 636 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:05:42.857539 631 server_base.cc:1034] running on GCE node
W20260923 15:05:42.857357 637 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:05:42.857867 631 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:05:42.858165 631 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:05:42.859342 631 hybrid_clock.cc:648] HybridClock initialized: now 1790175942859328 us; error 31 us; skew 500 ppm
Sep 23 15:05:42 dist-test-slave-cnb0 krb5kdc[32723](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175942, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.131@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:05:42.862069 631 init.cc:377] Logged in from keytab as kudu/127.13.75.131@KRBTEST.COM (short username kudu)
I20260923 15:05:42.862545 631 webserver.cc:460] Webserver started at http://127.13.75.131:42575/ using document root <none> and password file <none>
I20260923 15:05:42.862785 631 fs_manager.cc:359] Metadata directory not provided
I20260923 15:05:42.862854 631 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:05:42.863018 631 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:05:42.864058 631 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175691491353-13614-0/minicluster-data/ts-2/data/instance:
uuid: "d769b61d16e14eab958ae079252e8c1b"
format_stamp: "Formatted at 2026-09-23 15:05:42 on dist-test-slave-cnb0"
server_key: "83b5b80507954fc43f67286fb9ae7e15"
server_key_iv: "03eca7f8cc5c73a84a46aba4b44e9fda"
server_key_version: "encryptionkey@0"
I20260923 15:05:42.864431 631 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175691491353-13614-0/minicluster-data/ts-2/wal/instance:
uuid: "d769b61d16e14eab958ae079252e8c1b"
format_stamp: "Formatted at 2026-09-23 15:05:42 on dist-test-slave-cnb0"
server_key: "83b5b80507954fc43f67286fb9ae7e15"
server_key_iv: "03eca7f8cc5c73a84a46aba4b44e9fda"
server_key_version: "encryptionkey@0"
I20260923 15:05:42.865798 631 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260923 15:05:42.866637 646 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:05:42.866950 631 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260923 15:05:42.867063 631 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175691491353-13614-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175691491353-13614-0/minicluster-data/ts-2/wal
uuid: "d769b61d16e14eab958ae079252e8c1b"
format_stamp: "Formatted at 2026-09-23 15:05:42 on dist-test-slave-cnb0"
server_key: "83b5b80507954fc43f67286fb9ae7e15"
server_key_iv: "03eca7f8cc5c73a84a46aba4b44e9fda"
server_key_version: "encryptionkey@0"
I20260923 15:05:42.867146 631 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175691491353-13614-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175691491353-13614-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175691491353-13614-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:05:42.881201 631 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:05:42.883666 631 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:05:42.883838 631 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:05:42.884081 631 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 15:05:42.884405 631 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 15:05:42.884461 631 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 15:05:42.884505 631 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 15:05:42.884532 631 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 15:05:42.890194 631 rpc_server.cc:307] RPC server started. Bound to: 127.13.75.131:34749
I20260923 15:05:42.890290 759 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.75.131:34749 every 8 connection(s)
I20260923 15:05:42.890662 631 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175691491353-13614-0/minicluster-data/ts-2/data/info.pb
Sep 23 15:05:42 dist-test-slave-cnb0 krb5kdc[32723](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175942, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.131@KRBTEST.COM for kudu/127.13.75.190@KRBTEST.COM
I20260923 15:05:42.899418 13614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKC4soy/build/release/bin/kudu as pid 631
I20260923 15:05:42.899540 13614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175691491353-13614-0/minicluster-data/ts-2/wal/instance
I20260923 15:05:42.899731 13614 external_mini_cluster.cc:1428] Setting key a99f922f2dbf65ee154d02459384543f
I20260923 15:05:42.900255 346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:42.891464 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.131:45089 (local address 127.13.75.190:35527)
0923 15:05:42.891532 (+ 68us) server_negotiation.cc:206] Beginning negotiation
0923 15:05:42.891538 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 15:05:42.891781 (+ 243us) server_negotiation.cc:407] Connection header received
0923 15:05:42.892278 (+ 497us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:05:42.892279 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:05:42.892288 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:05:42.892350 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:05:42.893317 (+ 967us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:05:42.893815 (+ 498us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:05:42.894771 (+ 956us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:05:42.895050 (+ 279us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:05:42.895135 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:05:42.897200 (+ 2065us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:05:42.897202 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:05:42.897202 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:05:42.897208 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:05:42.898641 (+ 1433us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:05:42.899031 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:05:42.899032 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:05:42.899032 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:05:42.899085 (+ 53us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:05:42.899312 (+ 227us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:05:42.899313 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:05:42.899314 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:05:42.899468 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:05:42.899551 (+ 83us) server_negotiation.cc:1035] Waiting for connection context
0923 15:05:42.900025 (+ 474us) server_negotiation.cc:299] Negotiation successful
0923 15:05:42.900116 (+ 91us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260923 15:05:42.900496 762 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:42.891452 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:35527 (local address 127.13.75.131:45089)
0923 15:05:42.891642 (+ 190us) negotiation.cc:107] Waiting for socket to connect
0923 15:05:42.891687 (+ 45us) client_negotiation.cc:174] Beginning negotiation
0923 15:05:42.892169 (+ 482us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:05:42.892416 (+ 247us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:05:42.892419 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:05:42.893032 (+ 613us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:05:42.893231 (+ 199us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:05:42.893236 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:05:42.893899 (+ 663us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:05:42.893900 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:05:42.894690 (+ 790us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:05:42.894692 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:05:42.895156 (+ 464us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:05:42.895157 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:05:42.895286 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:05:42.895733 (+ 447us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:05:42.895740 (+ 7us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:05:42.897114 (+ 1374us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:05:42.898725 (+ 1611us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:05:42.898731 (+ 6us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:05:42.898733 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:05:42.898977 (+ 244us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:05:42.899144 (+ 167us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:05:42.899145 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:05:42.899145 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:05:42.899246 (+ 101us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:05:42.899575 (+ 329us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:05:42.899583 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:05:42.899810 (+ 227us) client_negotiation.cc:769] Sending connection context
0923 15:05:42.899986 (+ 176us) client_negotiation.cc:240] Negotiation successful
0923 15:05:42.900216 (+ 230us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"thread_start_us":88,"threads_started":1}
I20260923 15:05:42.901041 760 heartbeater.cc:347] Connected to a master server at 127.13.75.190:35527
I20260923 15:05:42.901146 760 heartbeater.cc:464] Registering TS with master...
I20260923 15:05:42.901307 760 heartbeater.cc:511] Master 127.13.75.190:35527 requested a full tablet report, sending...
I20260923 15:05:42.901669 300 ts_manager.cc:194] Registered new tserver with Master: d769b61d16e14eab958ae079252e8c1b (127.13.75.131:34749)
I20260923 15:05:42.902063 300 master_service.cc:496] Signed X509 certificate for tserver {username='kudu', principal='kudu/127.13.75.131@KRBTEST.COM'} at 127.13.75.131:45089
I20260923 15:05:42.911991 13614 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260923 15:05:42.920217 346 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:42.913447 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44212 (local address 127.13.75.190:35527)
0923 15:05:42.913559 (+ 112us) server_negotiation.cc:206] Beginning negotiation
0923 15:05:42.913561 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:05:42.913573 (+ 12us) server_negotiation.cc:407] Connection header received
0923 15:05:42.913645 (+ 72us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:05:42.913646 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:05:42.913657 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:05:42.913720 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:05:42.914320 (+ 600us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:05:42.914953 (+ 633us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:05:42.915734 (+ 781us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:05:42.915949 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:05:42.916010 (+ 61us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:05:42.917264 (+ 1254us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:05:42.917265 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:05:42.917265 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:05:42.917272 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:05:42.918940 (+ 1668us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:05:42.919310 (+ 370us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:05:42.919311 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:05:42.919311 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:05:42.919367 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:05:42.919593 (+ 226us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:05:42.919593 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:05:42.919594 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:05:42.919793 (+ 199us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:05:42.919891 (+ 98us) server_negotiation.cc:1035] Waiting for connection context
0923 15:05:42.920012 (+ 121us) server_negotiation.cc:299] Negotiation successful
0923 15:05:42.920081 (+ 69us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":79}
I20260923 15:05:42.922276 300 catalog_manager.cc:1908] Servicing CreateTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44212:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 15:05:42.924177 300 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 15:05:42.931527 774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:42.927096 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38012 (local address 127.13.75.129:45255)
0923 15:05:42.927524 (+ 428us) server_negotiation.cc:206] Beginning negotiation
0923 15:05:42.927530 (+ 6us) server_negotiation.cc:399] Waiting for connection header
0923 15:05:42.927542 (+ 12us) server_negotiation.cc:407] Connection header received
0923 15:05:42.927591 (+ 49us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:05:42.927594 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:05:42.927627 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:05:42.927720 (+ 93us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:05:42.927945 (+ 225us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:05:42.928818 (+ 873us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:05:42.930203 (+ 1385us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:05:42.931074 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:05:42.931148 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:05:42.931215 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0923 15:05:42.931380 (+ 165us) server_negotiation.cc:299] Negotiation successful
0923 15:05:42.931426 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":384,"thread_start_us":116,"threads_started":1}
I20260923 15:05:42.931519 771 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:42.927000 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.129:45255 (local address 127.0.0.1:38012)
0923 15:05:42.927313 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0923 15:05:42.927326 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0923 15:05:42.927381 (+ 55us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:05:42.927748 (+ 367us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:05:42.927752 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:05:42.927755 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:05:42.927840 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:05:42.927842 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:05:42.928930 (+ 1088us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:05:42.928932 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:05:42.930101 (+ 1169us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:05:42.930104 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:05:42.931162 (+ 1058us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:05:42.931163 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:05:42.931254 (+ 91us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:05:42.931270 (+ 16us) client_negotiation.cc:769] Sending connection context
0923 15:05:42.931359 (+ 89us) client_negotiation.cc:240] Negotiation successful
0923 15:05:42.931425 (+ 66us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":295,"thread_start_us":114,"threads_started":1}
I20260923 15:05:42.932403 775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:42.927275 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.130:44663 (local address 127.0.0.1:49228)
0923 15:05:42.927588 (+ 313us) negotiation.cc:107] Waiting for socket to connect
0923 15:05:42.927598 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 15:05:42.927666 (+ 68us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:05:42.928000 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:05:42.928001 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:05:42.928003 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:05:42.928051 (+ 48us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:05:42.928053 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:05:42.929530 (+ 1477us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:05:42.929531 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:05:42.930965 (+ 1434us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:05:42.930968 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:05:42.932122 (+ 1154us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:05:42.932123 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:05:42.932198 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:05:42.932202 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 15:05:42.932259 (+ 57us) client_negotiation.cc:240] Negotiation successful
0923 15:05:42.932311 (+ 52us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":292,"thread_start_us":84,"threads_started":1}
I20260923 15:05:42.932520 772 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:42.926997 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.131:34749 (local address 127.0.0.1:41764)
0923 15:05:42.927198 (+ 201us) negotiation.cc:107] Waiting for socket to connect
0923 15:05:42.927245 (+ 47us) client_negotiation.cc:174] Beginning negotiation
0923 15:05:42.927323 (+ 78us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:05:42.928070 (+ 747us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:05:42.928071 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:05:42.928074 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:05:42.928131 (+ 57us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:05:42.928133 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:05:42.929583 (+ 1450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:05:42.929585 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:05:42.930631 (+ 1046us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:05:42.930634 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:05:42.932259 (+ 1625us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:05:42.932260 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:05:42.932350 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:05:42.932354 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 15:05:42.932389 (+ 35us) client_negotiation.cc:240] Negotiation successful
0923 15:05:42.932435 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165,"mutex_wait_us":83,"thread_start_us":56,"threads_started":1}
I20260923 15:05:42.932457 426 tablet_service.cc:1467] Processing CreateTablet for tablet 8e9bd76fc2d04a0c8927558b23775c78 (DEFAULT_TABLE table=test-table [id=055f1e70a63f4533ab2b9d9e7d5fc867]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 15:05:42.932816 776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:42.927394 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49228 (local address 127.13.75.130:44663)
0923 15:05:42.927575 (+ 181us) server_negotiation.cc:206] Beginning negotiation
0923 15:05:42.927579 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 15:05:42.927664 (+ 85us) server_negotiation.cc:407] Connection header received
0923 15:05:42.927812 (+ 148us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:05:42.927816 (+ 4us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:05:42.927833 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:05:42.927916 (+ 83us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:05:42.928132 (+ 216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:05:42.929388 (+ 1256us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:05:42.931077 (+ 1689us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:05:42.931887 (+ 810us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:05:42.932015 (+ 128us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:05:42.932078 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0923 15:05:42.932681 (+ 603us) server_negotiation.cc:299] Negotiation successful
0923 15:05:42.932728 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":144,"thread_start_us":97,"threads_started":1}
I20260923 15:05:42.932878 426 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8e9bd76fc2d04a0c8927558b23775c78. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:05:42.932878 773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:42.927068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41764 (local address 127.13.75.131:34749)
0923 15:05:42.927885 (+ 817us) server_negotiation.cc:206] Beginning negotiation
0923 15:05:42.927889 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 15:05:42.927900 (+ 11us) server_negotiation.cc:407] Connection header received
0923 15:05:42.927941 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:05:42.927943 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:05:42.927961 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:05:42.928220 (+ 259us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:05:42.928302 (+ 82us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:05:42.929465 (+ 1163us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:05:42.930738 (+ 1273us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:05:42.931577 (+ 839us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:05:42.932616 (+ 1039us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:05:42.932702 (+ 86us) server_negotiation.cc:1035] Waiting for connection context
0923 15:05:42.932732 (+ 30us) server_negotiation.cc:299] Negotiation successful
0923 15:05:42.932780 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":768,"thread_start_us":108,"threads_started":1}
I20260923 15:05:42.933435 560 tablet_service.cc:1467] Processing CreateTablet for tablet 8e9bd76fc2d04a0c8927558b23775c78 (DEFAULT_TABLE table=test-table [id=055f1e70a63f4533ab2b9d9e7d5fc867]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 15:05:42.933523 694 tablet_service.cc:1467] Processing CreateTablet for tablet 8e9bd76fc2d04a0c8927558b23775c78 (DEFAULT_TABLE table=test-table [id=055f1e70a63f4533ab2b9d9e7d5fc867]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 15:05:42.933794 560 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8e9bd76fc2d04a0c8927558b23775c78. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:05:42.933880 694 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 8e9bd76fc2d04a0c8927558b23775c78. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:05:42.936785 777 tablet_bootstrap.cc:492] T 8e9bd76fc2d04a0c8927558b23775c78 P e2bd2678911741f0bf628d30ec52b3dc: Bootstrap starting.
I20260923 15:05:42.937186 778 tablet_bootstrap.cc:492] T 8e9bd76fc2d04a0c8927558b23775c78 P 12b7cc036e1d4585ac0afd587eaaeb78: Bootstrap starting.
I20260923 15:05:42.937551 779 tablet_bootstrap.cc:492] T 8e9bd76fc2d04a0c8927558b23775c78 P d769b61d16e14eab958ae079252e8c1b: Bootstrap starting.
I20260923 15:05:42.937633 777 tablet_bootstrap.cc:654] T 8e9bd76fc2d04a0c8927558b23775c78 P e2bd2678911741f0bf628d30ec52b3dc: Neither blocks nor log segments found. Creating new log.
I20260923 15:05:42.937942 778 tablet_bootstrap.cc:654] T 8e9bd76fc2d04a0c8927558b23775c78 P 12b7cc036e1d4585ac0afd587eaaeb78: Neither blocks nor log segments found. Creating new log.
I20260923 15:05:42.938287 779 tablet_bootstrap.cc:654] T 8e9bd76fc2d04a0c8927558b23775c78 P d769b61d16e14eab958ae079252e8c1b: Neither blocks nor log segments found. Creating new log.
I20260923 15:05:42.938311 777 log.cc:824] T 8e9bd76fc2d04a0c8927558b23775c78 P e2bd2678911741f0bf628d30ec52b3dc: Log is configured to *not* fsync() on all Append() calls
I20260923 15:05:42.938609 778 log.cc:824] T 8e9bd76fc2d04a0c8927558b23775c78 P 12b7cc036e1d4585ac0afd587eaaeb78: Log is configured to *not* fsync() on all Append() calls
I20260923 15:05:42.939030 779 log.cc:824] T 8e9bd76fc2d04a0c8927558b23775c78 P d769b61d16e14eab958ae079252e8c1b: Log is configured to *not* fsync() on all Append() calls
I20260923 15:05:42.939375 777 tablet_bootstrap.cc:492] T 8e9bd76fc2d04a0c8927558b23775c78 P e2bd2678911741f0bf628d30ec52b3dc: No bootstrap required, opened a new log
I20260923 15:05:42.939427 777 ts_tablet_manager.cc:1397] T 8e9bd76fc2d04a0c8927558b23775c78 P e2bd2678911741f0bf628d30ec52b3dc: Time spent bootstrapping tablet: real 0.003s user 0.002s sys 0.000s
I20260923 15:05:42.939493 778 tablet_bootstrap.cc:492] T 8e9bd76fc2d04a0c8927558b23775c78 P 12b7cc036e1d4585ac0afd587eaaeb78: No bootstrap required, opened a new log
I20260923 15:05:42.939561 778 ts_tablet_manager.cc:1397] T 8e9bd76fc2d04a0c8927558b23775c78 P 12b7cc036e1d4585ac0afd587eaaeb78: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260923 15:05:42.939888 779 tablet_bootstrap.cc:492] T 8e9bd76fc2d04a0c8927558b23775c78 P d769b61d16e14eab958ae079252e8c1b: No bootstrap required, opened a new log
I20260923 15:05:42.939949 779 ts_tablet_manager.cc:1397] T 8e9bd76fc2d04a0c8927558b23775c78 P d769b61d16e14eab958ae079252e8c1b: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260923 15:05:42.941066 778 raft_consensus.cc:348] T 8e9bd76fc2d04a0c8927558b23775c78 P 12b7cc036e1d4585ac0afd587eaaeb78 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12b7cc036e1d4585ac0afd587eaaeb78" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 45255 } } peers { permanent_uuid: "d769b61d16e14eab958ae079252e8c1b" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 34749 } } peers { permanent_uuid: "e2bd2678911741f0bf628d30ec52b3dc" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 44663 } }
I20260923 15:05:42.941205 778 raft_consensus.cc:374] T 8e9bd76fc2d04a0c8927558b23775c78 P 12b7cc036e1d4585ac0afd587eaaeb78 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:05:42.941228 778 raft_consensus.cc:729] T 8e9bd76fc2d04a0c8927558b23775c78 P 12b7cc036e1d4585ac0afd587eaaeb78 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 12b7cc036e1d4585ac0afd587eaaeb78, State: Initialized, Role: FOLLOWER
I20260923 15:05:42.941363 778 consensus_queue.cc:260] T 8e9bd76fc2d04a0c8927558b23775c78 P 12b7cc036e1d4585ac0afd587eaaeb78 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12b7cc036e1d4585ac0afd587eaaeb78" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 45255 } } peers { permanent_uuid: "d769b61d16e14eab958ae079252e8c1b" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 34749 } } peers { permanent_uuid: "e2bd2678911741f0bf628d30ec52b3dc" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 44663 } }
I20260923 15:05:42.941674 778 ts_tablet_manager.cc:1428] T 8e9bd76fc2d04a0c8927558b23775c78 P 12b7cc036e1d4585ac0afd587eaaeb78: Time spent starting tablet: real 0.002s user 0.002s sys 0.000s
I20260923 15:05:42.941672 492 heartbeater.cc:503] Master 127.13.75.190:35527 was elected leader, sending a full tablet report...
I20260923 15:05:42.941665 777 raft_consensus.cc:348] T 8e9bd76fc2d04a0c8927558b23775c78 P e2bd2678911741f0bf628d30ec52b3dc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12b7cc036e1d4585ac0afd587eaaeb78" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 45255 } } peers { permanent_uuid: "d769b61d16e14eab958ae079252e8c1b" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 34749 } } peers { permanent_uuid: "e2bd2678911741f0bf628d30ec52b3dc" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 44663 } }
I20260923 15:05:42.941845 777 raft_consensus.cc:374] T 8e9bd76fc2d04a0c8927558b23775c78 P e2bd2678911741f0bf628d30ec52b3dc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:05:42.941880 777 raft_consensus.cc:729] T 8e9bd76fc2d04a0c8927558b23775c78 P e2bd2678911741f0bf628d30ec52b3dc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e2bd2678911741f0bf628d30ec52b3dc, State: Initialized, Role: FOLLOWER
I20260923 15:05:42.942023 777 consensus_queue.cc:260] T 8e9bd76fc2d04a0c8927558b23775c78 P e2bd2678911741f0bf628d30ec52b3dc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12b7cc036e1d4585ac0afd587eaaeb78" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 45255 } } peers { permanent_uuid: "d769b61d16e14eab958ae079252e8c1b" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 34749 } } peers { permanent_uuid: "e2bd2678911741f0bf628d30ec52b3dc" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 44663 } }
I20260923 15:05:42.942063 779 raft_consensus.cc:348] T 8e9bd76fc2d04a0c8927558b23775c78 P d769b61d16e14eab958ae079252e8c1b [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12b7cc036e1d4585ac0afd587eaaeb78" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 45255 } } peers { permanent_uuid: "d769b61d16e14eab958ae079252e8c1b" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 34749 } } peers { permanent_uuid: "e2bd2678911741f0bf628d30ec52b3dc" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 44663 } }
I20260923 15:05:42.942242 779 raft_consensus.cc:374] T 8e9bd76fc2d04a0c8927558b23775c78 P d769b61d16e14eab958ae079252e8c1b [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:05:42.942279 779 raft_consensus.cc:729] T 8e9bd76fc2d04a0c8927558b23775c78 P d769b61d16e14eab958ae079252e8c1b [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: d769b61d16e14eab958ae079252e8c1b, State: Initialized, Role: FOLLOWER
I20260923 15:05:42.942392 779 consensus_queue.cc:260] T 8e9bd76fc2d04a0c8927558b23775c78 P d769b61d16e14eab958ae079252e8c1b [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12b7cc036e1d4585ac0afd587eaaeb78" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 45255 } } peers { permanent_uuid: "d769b61d16e14eab958ae079252e8c1b" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 34749 } } peers { permanent_uuid: "e2bd2678911741f0bf628d30ec52b3dc" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 44663 } }
I20260923 15:05:42.942683 779 ts_tablet_manager.cc:1428] T 8e9bd76fc2d04a0c8927558b23775c78 P d769b61d16e14eab958ae079252e8c1b: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260923 15:05:42.942701 760 heartbeater.cc:503] Master 127.13.75.190:35527 was elected leader, sending a full tablet report...
I20260923 15:05:42.943290 777 ts_tablet_manager.cc:1428] T 8e9bd76fc2d04a0c8927558b23775c78 P e2bd2678911741f0bf628d30ec52b3dc: Time spent starting tablet: real 0.004s user 0.003s sys 0.000s
I20260923 15:05:42.943440 626 heartbeater.cc:503] Master 127.13.75.190:35527 was elected leader, sending a full tablet report...
W20260923 15:05:42.974056 627 tablet.cc:2369] T 8e9bd76fc2d04a0c8927558b23775c78 P e2bd2678911741f0bf628d30ec52b3dc: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260923 15:05:43.050143 493 tablet.cc:2369] T 8e9bd76fc2d04a0c8927558b23775c78 P 12b7cc036e1d4585ac0afd587eaaeb78: Can't schedule compaction. Clean time has not been advanced past its initial value.
W20260923 15:05:43.141319 761 tablet.cc:2369] T 8e9bd76fc2d04a0c8927558b23775c78 P d769b61d16e14eab958ae079252e8c1b: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 15:05:43.192196 783 raft_consensus.cc:482] T 8e9bd76fc2d04a0c8927558b23775c78 P 12b7cc036e1d4585ac0afd587eaaeb78 [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 15:05:43.192334 783 raft_consensus.cc:504] T 8e9bd76fc2d04a0c8927558b23775c78 P 12b7cc036e1d4585ac0afd587eaaeb78 [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12b7cc036e1d4585ac0afd587eaaeb78" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 45255 } } peers { permanent_uuid: "d769b61d16e14eab958ae079252e8c1b" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 34749 } } peers { permanent_uuid: "e2bd2678911741f0bf628d30ec52b3dc" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 44663 } }
I20260923 15:05:43.192793 783 leader_election.cc:288] T 8e9bd76fc2d04a0c8927558b23775c78 P 12b7cc036e1d4585ac0afd587eaaeb78 [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers d769b61d16e14eab958ae079252e8c1b (127.13.75.131:34749), e2bd2678911741f0bf628d30ec52b3dc (127.13.75.130:44663)
I20260923 15:05:43.195999 776 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:43.193264 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.129:33727 (local address 127.13.75.130:44663)
0923 15:05:43.193337 (+ 73us) server_negotiation.cc:206] Beginning negotiation
0923 15:05:43.193340 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:05:43.193411 (+ 71us) server_negotiation.cc:407] Connection header received
0923 15:05:43.193459 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:05:43.193459 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:05:43.193471 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:05:43.193526 (+ 55us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:05:43.193707 (+ 181us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:05:43.194190 (+ 483us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:05:43.195237 (+ 1047us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:05:43.195628 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:05:43.195716 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:05:43.195726 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0923 15:05:43.195883 (+ 157us) server_negotiation.cc:299] Negotiation successful
0923 15:05:43.195922 (+ 39us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260923 15:05:43.196048 787 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:43.193171 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.130:44663 (local address 127.13.75.129:33727)
0923 15:05:43.193332 (+ 161us) negotiation.cc:107] Waiting for socket to connect
0923 15:05:43.193341 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 15:05:43.193408 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:05:43.193540 (+ 132us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:05:43.193540 (+ 0us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:05:43.193542 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:05:43.193628 (+ 86us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:05:43.193629 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:05:43.194304 (+ 675us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:05:43.194305 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:05:43.195139 (+ 834us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:05:43.195142 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:05:43.195716 (+ 574us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:05:43.195717 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:05:43.195792 (+ 75us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:05:43.195796 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 15:05:43.195873 (+ 77us) client_negotiation.cc:240] Negotiation successful
0923 15:05:43.195917 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":141,"thread_start_us":57,"threads_started":1}
I20260923 15:05:43.196053 773 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:43.193113 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.129:60141 (local address 127.13.75.131:34749)
0923 15:05:43.193177 (+ 64us) server_negotiation.cc:206] Beginning negotiation
0923 15:05:43.193179 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:05:43.193193 (+ 14us) server_negotiation.cc:407] Connection header received
0923 15:05:43.193298 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:05:43.193301 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:05:43.193315 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:05:43.193351 (+ 36us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:05:43.193605 (+ 254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:05:43.194076 (+ 471us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:05:43.195224 (+ 1148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:05:43.195650 (+ 426us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:05:43.195765 (+ 115us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:05:43.195774 (+ 9us) server_negotiation.cc:1035] Waiting for connection context
0923 15:05:43.195924 (+ 150us) server_negotiation.cc:299] Negotiation successful
0923 15:05:43.195960 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":38}
I20260923 15:05:43.196065 786 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:43.192977 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.131:34749 (local address 127.13.75.129:60141)
0923 15:05:43.193112 (+ 135us) negotiation.cc:107] Waiting for socket to connect
0923 15:05:43.193125 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0923 15:05:43.193182 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:05:43.193456 (+ 274us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:05:43.193457 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:05:43.193459 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:05:43.193541 (+ 82us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:05:43.193543 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:05:43.194162 (+ 619us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:05:43.194163 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:05:43.195122 (+ 959us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:05:43.195125 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:05:43.195767 (+ 642us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:05:43.195768 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:05:43.195844 (+ 76us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:05:43.195848 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 15:05:43.195919 (+ 71us) client_negotiation.cc:240] Negotiation successful
0923 15:05:43.195960 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":116,"mutex_wait_us":51,"thread_start_us":87,"threads_started":1}
I20260923 15:05:43.196552 714 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e9bd76fc2d04a0c8927558b23775c78" candidate_uuid: "12b7cc036e1d4585ac0afd587eaaeb78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d769b61d16e14eab958ae079252e8c1b" is_pre_election: true
I20260923 15:05:43.196552 580 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e9bd76fc2d04a0c8927558b23775c78" candidate_uuid: "12b7cc036e1d4585ac0afd587eaaeb78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2bd2678911741f0bf628d30ec52b3dc" is_pre_election: true
I20260923 15:05:43.196713 580 raft_consensus.cc:2454] T 8e9bd76fc2d04a0c8927558b23775c78 P e2bd2678911741f0bf628d30ec52b3dc [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12b7cc036e1d4585ac0afd587eaaeb78 in term 0.
I20260923 15:05:43.196713 714 raft_consensus.cc:2454] T 8e9bd76fc2d04a0c8927558b23775c78 P d769b61d16e14eab958ae079252e8c1b [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate 12b7cc036e1d4585ac0afd587eaaeb78 in term 0.
I20260923 15:05:43.196960 381 leader_election.cc:302] T 8e9bd76fc2d04a0c8927558b23775c78 P 12b7cc036e1d4585ac0afd587eaaeb78 [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12b7cc036e1d4585ac0afd587eaaeb78, e2bd2678911741f0bf628d30ec52b3dc; no voters:
I20260923 15:05:43.197124 783 raft_consensus.cc:2781] T 8e9bd76fc2d04a0c8927558b23775c78 P 12b7cc036e1d4585ac0afd587eaaeb78 [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 15:05:43.197193 783 raft_consensus.cc:482] T 8e9bd76fc2d04a0c8927558b23775c78 P 12b7cc036e1d4585ac0afd587eaaeb78 [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
I20260923 15:05:43.197211 783 raft_consensus.cc:3037] T 8e9bd76fc2d04a0c8927558b23775c78 P 12b7cc036e1d4585ac0afd587eaaeb78 [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:05:43.198005 783 raft_consensus.cc:504] T 8e9bd76fc2d04a0c8927558b23775c78 P 12b7cc036e1d4585ac0afd587eaaeb78 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12b7cc036e1d4585ac0afd587eaaeb78" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 45255 } } peers { permanent_uuid: "d769b61d16e14eab958ae079252e8c1b" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 34749 } } peers { permanent_uuid: "e2bd2678911741f0bf628d30ec52b3dc" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 44663 } }
I20260923 15:05:43.198149 783 leader_election.cc:288] T 8e9bd76fc2d04a0c8927558b23775c78 P 12b7cc036e1d4585ac0afd587eaaeb78 [CANDIDATE]: Term 1 election: Requested vote from peers d769b61d16e14eab958ae079252e8c1b (127.13.75.131:34749), e2bd2678911741f0bf628d30ec52b3dc (127.13.75.130:44663)
I20260923 15:05:43.198381 714 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e9bd76fc2d04a0c8927558b23775c78" candidate_uuid: "12b7cc036e1d4585ac0afd587eaaeb78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "d769b61d16e14eab958ae079252e8c1b"
I20260923 15:05:43.198438 580 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "8e9bd76fc2d04a0c8927558b23775c78" candidate_uuid: "12b7cc036e1d4585ac0afd587eaaeb78" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "e2bd2678911741f0bf628d30ec52b3dc"
I20260923 15:05:43.198465 714 raft_consensus.cc:3037] T 8e9bd76fc2d04a0c8927558b23775c78 P d769b61d16e14eab958ae079252e8c1b [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:05:43.198540 580 raft_consensus.cc:3037] T 8e9bd76fc2d04a0c8927558b23775c78 P e2bd2678911741f0bf628d30ec52b3dc [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:05:43.199329 714 raft_consensus.cc:2454] T 8e9bd76fc2d04a0c8927558b23775c78 P d769b61d16e14eab958ae079252e8c1b [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12b7cc036e1d4585ac0afd587eaaeb78 in term 1.
I20260923 15:05:43.199347 580 raft_consensus.cc:2454] T 8e9bd76fc2d04a0c8927558b23775c78 P e2bd2678911741f0bf628d30ec52b3dc [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 12b7cc036e1d4585ac0afd587eaaeb78 in term 1.
I20260923 15:05:43.199529 381 leader_election.cc:302] T 8e9bd76fc2d04a0c8927558b23775c78 P 12b7cc036e1d4585ac0afd587eaaeb78 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 12b7cc036e1d4585ac0afd587eaaeb78, e2bd2678911741f0bf628d30ec52b3dc; no voters:
I20260923 15:05:43.199640 783 raft_consensus.cc:2781] T 8e9bd76fc2d04a0c8927558b23775c78 P 12b7cc036e1d4585ac0afd587eaaeb78 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 15:05:43.199790 783 raft_consensus.cc:686] T 8e9bd76fc2d04a0c8927558b23775c78 P 12b7cc036e1d4585ac0afd587eaaeb78 [term 1 LEADER]: Becoming Leader. State: Replica: 12b7cc036e1d4585ac0afd587eaaeb78, State: Running, Role: LEADER
I20260923 15:05:43.199868 783 consensus_queue.cc:237] T 8e9bd76fc2d04a0c8927558b23775c78 P 12b7cc036e1d4585ac0afd587eaaeb78 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12b7cc036e1d4585ac0afd587eaaeb78" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 45255 } } peers { permanent_uuid: "d769b61d16e14eab958ae079252e8c1b" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 34749 } } peers { permanent_uuid: "e2bd2678911741f0bf628d30ec52b3dc" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 44663 } }
I20260923 15:05:43.200829 301 catalog_manager.cc:5495] T 8e9bd76fc2d04a0c8927558b23775c78 P 12b7cc036e1d4585ac0afd587eaaeb78 reported cstate change: term changed from 0 to 1, leader changed from <none> to 12b7cc036e1d4585ac0afd587eaaeb78 (127.13.75.129). New cstate: current_term: 1 leader_uuid: "12b7cc036e1d4585ac0afd587eaaeb78" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "12b7cc036e1d4585ac0afd587eaaeb78" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 45255 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "d769b61d16e14eab958ae079252e8c1b" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 34749 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "e2bd2678911741f0bf628d30ec52b3dc" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 44663 } health_report { overall_health: UNKNOWN } } }
I20260923 15:05:43.216895 774 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:43.214084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38020 (local address 127.13.75.129:45255)
0923 15:05:43.214187 (+ 103us) server_negotiation.cc:206] Beginning negotiation
0923 15:05:43.214189 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:05:43.214199 (+ 10us) server_negotiation.cc:407] Connection header received
0923 15:05:43.214241 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:05:43.214242 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:05:43.214265 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:05:43.214344 (+ 79us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 15:05:43.214486 (+ 142us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:05:43.215036 (+ 550us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:05:43.215942 (+ 906us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:05:43.216181 (+ 239us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:05:43.216248 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:05:43.216498 (+ 250us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 15:05:43.216618 (+ 120us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 15:05:43.216700 (+ 82us) server_negotiation.cc:1035] Waiting for connection context
0923 15:05:43.216749 (+ 49us) server_negotiation.cc:299] Negotiation successful
0923 15:05:43.216770 (+ 21us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":65}
I20260923 15:05:43.219066 714 raft_consensus.cc:1261] T 8e9bd76fc2d04a0c8927558b23775c78 P d769b61d16e14eab958ae079252e8c1b [term 1 FOLLOWER]: Refusing update from remote peer 12b7cc036e1d4585ac0afd587eaaeb78: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 15:05:43.219079 580 raft_consensus.cc:1261] T 8e9bd76fc2d04a0c8927558b23775c78 P e2bd2678911741f0bf628d30ec52b3dc [term 1 FOLLOWER]: Refusing update from remote peer 12b7cc036e1d4585ac0afd587eaaeb78: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 15:05:43.219511 783 consensus_queue.cc:1035] T 8e9bd76fc2d04a0c8927558b23775c78 P 12b7cc036e1d4585ac0afd587eaaeb78 [LEADER]: Connected to new peer: Peer: permanent_uuid: "e2bd2678911741f0bf628d30ec52b3dc" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 44663 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 15:05:43.219717 788 consensus_queue.cc:1035] T 8e9bd76fc2d04a0c8927558b23775c78 P 12b7cc036e1d4585ac0afd587eaaeb78 [LEADER]: Connected to new peer: Peer: permanent_uuid: "d769b61d16e14eab958ae079252e8c1b" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 34749 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 15:05:43.222728 791 mvcc.cc:205] Tried to move back new op lower bound from 7332560663421472768 to 7332560663347494912. Current Snapshot: MvccSnapshot[applied={T|T < 7332560663421472768 or (T in {7332560663421472768})}]
I20260923 15:05:43.223330 792 mvcc.cc:205] Tried to move back new op lower bound from 7332560663421472768 to 7332560663347494912. Current Snapshot: MvccSnapshot[applied={T|T < 7332560663421472768 or (T in {7332560663421472768})}]
I20260923 15:05:43.224593 32767 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44212:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 15:05:43.224715 32767 catalog_manager.cc:2645] Servicing DeleteTable request from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:44212:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 15:05:43.225565 32767 catalog_manager.cc:5782] T 00000000000000000000000000000000 P 137f102e9b8b460f94101de0862c072c: Sending DeleteTablet for 3 replicas of tablet 8e9bd76fc2d04a0c8927558b23775c78
I20260923 15:05:43.226014 426 tablet_service.cc:1514] Processing DeleteTablet for tablet 8e9bd76fc2d04a0c8927558b23775c78 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 15:05:43 UTC) from {username='kudu', principal='kudu/127.13.75.190@KRBTEST.COM'} at 127.0.0.1:38012
I20260923 15:05:43.226291 801 tablet_replica.cc:330] T 8e9bd76fc2d04a0c8927558b23775c78 P 12b7cc036e1d4585ac0afd587eaaeb78: stopping tablet replica
I20260923 15:05:43.226280 694 tablet_service.cc:1514] Processing DeleteTablet for tablet 8e9bd76fc2d04a0c8927558b23775c78 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 15:05:43 UTC) from {username='kudu', principal='kudu/127.13.75.190@KRBTEST.COM'} at 127.0.0.1:41764
I20260923 15:05:43.226398 801 raft_consensus.cc:2229] T 8e9bd76fc2d04a0c8927558b23775c78 P 12b7cc036e1d4585ac0afd587eaaeb78 [term 1 LEADER]: Raft consensus shutting down.
I20260923 15:05:43.226531 802 tablet_replica.cc:330] T 8e9bd76fc2d04a0c8927558b23775c78 P d769b61d16e14eab958ae079252e8c1b: stopping tablet replica
I20260923 15:05:43.226516 560 tablet_service.cc:1514] Processing DeleteTablet for tablet 8e9bd76fc2d04a0c8927558b23775c78 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 15:05:43 UTC) from {username='kudu', principal='kudu/127.13.75.190@KRBTEST.COM'} at 127.0.0.1:49228
I20260923 15:05:43.226583 801 raft_consensus.cc:2258] T 8e9bd76fc2d04a0c8927558b23775c78 P 12b7cc036e1d4585ac0afd587eaaeb78 [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 15:05:43.226723 802 raft_consensus.cc:2229] T 8e9bd76fc2d04a0c8927558b23775c78 P d769b61d16e14eab958ae079252e8c1b [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 15:05:43.226845 13614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKC4soy/build/release/bin/kudu with pid 363
I20260923 15:05:43.226948 802 raft_consensus.cc:2258] T 8e9bd76fc2d04a0c8927558b23775c78 P d769b61d16e14eab958ae079252e8c1b [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 15:05:43.227474 803 tablet_replica.cc:330] T 8e9bd76fc2d04a0c8927558b23775c78 P e2bd2678911741f0bf628d30ec52b3dc: stopping tablet replica
I20260923 15:05:43.227625 802 ts_tablet_manager.cc:1905] T 8e9bd76fc2d04a0c8927558b23775c78 P d769b61d16e14eab958ae079252e8c1b: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 15:05:43.227672 803 raft_consensus.cc:2229] T 8e9bd76fc2d04a0c8927558b23775c78 P e2bd2678911741f0bf628d30ec52b3dc [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 15:05:43.227813 803 raft_consensus.cc:2258] T 8e9bd76fc2d04a0c8927558b23775c78 P e2bd2678911741f0bf628d30ec52b3dc [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 15:05:43.228416 803 ts_tablet_manager.cc:1905] T 8e9bd76fc2d04a0c8927558b23775c78 P e2bd2678911741f0bf628d30ec52b3dc: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 15:05:43.229923 802 ts_tablet_manager.cc:1918] T 8e9bd76fc2d04a0c8927558b23775c78 P d769b61d16e14eab958ae079252e8c1b: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 15:05:43.230007 802 log.cc:1196] T 8e9bd76fc2d04a0c8927558b23775c78 P d769b61d16e14eab958ae079252e8c1b: Deleting WAL directory at /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175691491353-13614-0/minicluster-data/ts-2/wal/wals/8e9bd76fc2d04a0c8927558b23775c78
I20260923 15:05:43.230242 803 ts_tablet_manager.cc:1918] T 8e9bd76fc2d04a0c8927558b23775c78 P e2bd2678911741f0bf628d30ec52b3dc: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 15:05:43.230332 803 log.cc:1196] T 8e9bd76fc2d04a0c8927558b23775c78 P e2bd2678911741f0bf628d30ec52b3dc: Deleting WAL directory at /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_4.1790175691491353-13614-0/minicluster-data/ts-1/wal/wals/8e9bd76fc2d04a0c8927558b23775c78
I20260923 15:05:43.230382 802 ts_tablet_manager.cc:1939] T 8e9bd76fc2d04a0c8927558b23775c78 P d769b61d16e14eab958ae079252e8c1b: Deleting consensus metadata
I20260923 15:05:43.230651 803 ts_tablet_manager.cc:1939] T 8e9bd76fc2d04a0c8927558b23775c78 P e2bd2678911741f0bf628d30ec52b3dc: Deleting consensus metadata
I20260923 15:05:43.230935 32755 catalog_manager.cc:4841] TS d769b61d16e14eab958ae079252e8c1b (127.13.75.131:34749): tablet 8e9bd76fc2d04a0c8927558b23775c78 (table test-table [id=055f1e70a63f4533ab2b9d9e7d5fc867]) successfully deleted
I20260923 15:05:43.231099 32755 catalog_manager.cc:4841] TS e2bd2678911741f0bf628d30ec52b3dc (127.13.75.130:44663): tablet 8e9bd76fc2d04a0c8927558b23775c78 (table test-table [id=055f1e70a63f4533ab2b9d9e7d5fc867]) successfully deleted
W20260923 15:05:43.233773 32754 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.75.129:45255 (error 108)
I20260923 15:05:43.234157 13614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKC4soy/build/release/bin/kudu with pid 497
I20260923 15:05:43.234532 775 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:43.234358 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.129:45255 (local address 127.0.0.1:38024)
0923 15:05:43.234448 (+ 90us) negotiation.cc:107] Waiting for socket to connect
0923 15:05:43.234490 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.129:45255: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":69}
W20260923 15:05:43.234826 32754 catalog_manager.cc:4584] TS 12b7cc036e1d4585ac0afd587eaaeb78 (127.13.75.129:45255): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet 8e9bd76fc2d04a0c8927558b23775c78: Network error: Client connection negotiation failed: client connection to 127.13.75.129:45255: connect: Connection refused (error 111)
I20260923 15:05:43.239624 13614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKC4soy/build/release/bin/kudu with pid 631
I20260923 15:05:43.244696 13614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKC4soy/build/release/bin/kudu with pid 32736
2026-09-23T15:05:43Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/4 (2186 ms)
[ RUN ] EncryptionPolicyTest.TestEncryptionPolicy/5
2026-09-23T15:05:43Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T15:05:43Z Disabled control of system clock
I20260923 15:05:43.274758 13614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKC4soy/build/release/bin/kudu
/tmp/dist-test-taskKC4soy/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790175691491353-13614-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790175691491353-13614-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790175691491353-13614-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790175691491353-13614-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.75.190:37493
--webserver_interface=127.13.75.190
--webserver_port=0
--builtin_ntp_servers=127.13.75.148:41433
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.75.190:37493
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260923 15:05:43.360731 808 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:05:43.360980 808 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:05:43.361003 808 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:05:43.362280 808 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 15:05:43.362336 808 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:05:43.362349 808 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:05:43.362362 808 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 15:05:43.362375 808 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 15:05:43.363857 808 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.75.148:41433
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790175691491353-13614-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790175691491353-13614-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.75.190:37493
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.75.190:37493
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790175691491353-13614-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.75.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.808
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790175691491353-13614-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:05:43.364158 808 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:05:43.364490 808 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:05:43.369690 816 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:05:43.369678 813 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:05:43.369681 814 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:05:43.370026 808 server_base.cc:1034] running on GCE node
I20260923 15:05:43.370280 808 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:05:43.370585 808 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:05:43.371778 808 hybrid_clock.cc:648] HybridClock initialized: now 1790175943371715 us; error 60 us; skew 500 ppm
I20260923 15:05:43.373179 808 webserver.cc:460] Webserver started at http://127.13.75.190:38683/ using document root <none> and password file <none>
I20260923 15:05:43.373456 808 fs_manager.cc:359] Metadata directory not provided
I20260923 15:05:43.373530 808 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:05:43.373687 808 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:05:43.374796 808 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790175691491353-13614-0/minicluster-data/master-0/data/instance:
uuid: "e89ecef503384f98958d1067d5b7a1c9"
format_stamp: "Formatted at 2026-09-23 15:05:43 on dist-test-slave-cnb0"
server_key: "ee4b7b4cb6f8c490bb3bfb927ff8c2ca"
server_key_iv: "3fa702ca6d898ce9522476a2c5d0de0d"
server_key_version: "encryptionkey@0"
I20260923 15:05:43.375236 808 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790175691491353-13614-0/minicluster-data/master-0/wal/instance:
uuid: "e89ecef503384f98958d1067d5b7a1c9"
format_stamp: "Formatted at 2026-09-23 15:05:43 on dist-test-slave-cnb0"
server_key: "ee4b7b4cb6f8c490bb3bfb927ff8c2ca"
server_key_iv: "3fa702ca6d898ce9522476a2c5d0de0d"
server_key_version: "encryptionkey@0"
I20260923 15:05:43.376605 808 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:05:43.377445 822 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:05:43.377642 808 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:05:43.377753 808 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790175691491353-13614-0/minicluster-data/master-0/data,/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790175691491353-13614-0/minicluster-data/master-0/wal
uuid: "e89ecef503384f98958d1067d5b7a1c9"
format_stamp: "Formatted at 2026-09-23 15:05:43 on dist-test-slave-cnb0"
server_key: "ee4b7b4cb6f8c490bb3bfb927ff8c2ca"
server_key_iv: "3fa702ca6d898ce9522476a2c5d0de0d"
server_key_version: "encryptionkey@0"
I20260923 15:05:43.377859 808 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790175691491353-13614-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790175691491353-13614-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790175691491353-13614-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:05:43.402393 808 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:05:43.402803 808 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:05:43.403029 808 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:05:43.407760 808 rpc_server.cc:307] RPC server started. Bound to: 127.13.75.190:37493
I20260923 15:05:43.407795 874 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.75.190:37493 every 8 connection(s)
I20260923 15:05:43.408206 808 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790175691491353-13614-0/minicluster-data/master-0/data/info.pb
I20260923 15:05:43.409226 875 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:05:43.409845 13614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKC4soy/build/release/bin/kudu as pid 808
I20260923 15:05:43.409979 13614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790175691491353-13614-0/minicluster-data/master-0/wal/instance
I20260923 15:05:43.410207 13614 external_mini_cluster.cc:1428] Setting key c46151669cd2eeba9111d1b855d2e8e0
I20260923 15:05:43.412554 875 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e89ecef503384f98958d1067d5b7a1c9: Bootstrap starting.
I20260923 15:05:43.413228 875 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e89ecef503384f98958d1067d5b7a1c9: Neither blocks nor log segments found. Creating new log.
I20260923 15:05:43.413553 875 log.cc:824] T 00000000000000000000000000000000 P e89ecef503384f98958d1067d5b7a1c9: Log is configured to *not* fsync() on all Append() calls
I20260923 15:05:43.414345 875 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e89ecef503384f98958d1067d5b7a1c9: No bootstrap required, opened a new log
I20260923 15:05:43.416513 878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:43.410813 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45944 (local address 127.13.75.190:37493)
0923 15:05:43.411051 (+ 238us) server_negotiation.cc:206] Beginning negotiation
0923 15:05:43.411059 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0923 15:05:43.411087 (+ 28us) server_negotiation.cc:407] Connection header received
0923 15:05:43.411600 (+ 513us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:05:43.411608 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:05:43.411677 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:05:43.411801 (+ 124us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:05:43.412078 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:05:43.413097 (+ 1019us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:05:43.414190 (+ 1093us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:05:43.414702 (+ 512us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:05:43.414804 (+ 102us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:05:43.415758 (+ 954us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:05:43.415762 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:05:43.415762 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 15:05:43.415938 (+ 176us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:05:43.415973 (+ 35us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 15:05:43.415984 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:05:43.416042 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0923 15:05:43.416168 (+ 126us) server_negotiation.cc:299] Negotiation successful
0923 15:05:43.416294 (+ 126us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":152,"thread_start_us":116,"threads_started":1}
I20260923 15:05:43.416779 875 raft_consensus.cc:348] T 00000000000000000000000000000000 P e89ecef503384f98958d1067d5b7a1c9 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e89ecef503384f98958d1067d5b7a1c9" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 37493 } }
I20260923 15:05:43.416963 875 raft_consensus.cc:374] T 00000000000000000000000000000000 P e89ecef503384f98958d1067d5b7a1c9 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:05:43.417016 875 raft_consensus.cc:729] T 00000000000000000000000000000000 P e89ecef503384f98958d1067d5b7a1c9 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e89ecef503384f98958d1067d5b7a1c9, State: Initialized, Role: FOLLOWER
I20260923 15:05:43.417145 875 consensus_queue.cc:260] T 00000000000000000000000000000000 P e89ecef503384f98958d1067d5b7a1c9 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e89ecef503384f98958d1067d5b7a1c9" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 37493 } }
I20260923 15:05:43.417232 875 raft_consensus.cc:388] T 00000000000000000000000000000000 P e89ecef503384f98958d1067d5b7a1c9 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 15:05:43.417291 875 raft_consensus.cc:482] T 00000000000000000000000000000000 P e89ecef503384f98958d1067d5b7a1c9 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 15:05:43.417346 875 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e89ecef503384f98958d1067d5b7a1c9 [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:05:43.417932 875 raft_consensus.cc:504] T 00000000000000000000000000000000 P e89ecef503384f98958d1067d5b7a1c9 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e89ecef503384f98958d1067d5b7a1c9" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 37493 } }
I20260923 15:05:43.418051 875 leader_election.cc:302] T 00000000000000000000000000000000 P e89ecef503384f98958d1067d5b7a1c9 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e89ecef503384f98958d1067d5b7a1c9; no voters:
I20260923 15:05:43.418278 875 leader_election.cc:288] T 00000000000000000000000000000000 P e89ecef503384f98958d1067d5b7a1c9 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 15:05:43.418358 880 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e89ecef503384f98958d1067d5b7a1c9 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 15:05:43.418568 880 raft_consensus.cc:686] T 00000000000000000000000000000000 P e89ecef503384f98958d1067d5b7a1c9 [term 1 LEADER]: Becoming Leader. State: Replica: e89ecef503384f98958d1067d5b7a1c9, State: Running, Role: LEADER
I20260923 15:05:43.418581 875 sys_catalog.cc:564] T 00000000000000000000000000000000 P e89ecef503384f98958d1067d5b7a1c9 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 15:05:43.418681 880 consensus_queue.cc:237] T 00000000000000000000000000000000 P e89ecef503384f98958d1067d5b7a1c9 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e89ecef503384f98958d1067d5b7a1c9" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 37493 } }
I20260923 15:05:43.419116 881 sys_catalog.cc:455] T 00000000000000000000000000000000 P e89ecef503384f98958d1067d5b7a1c9 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e89ecef503384f98958d1067d5b7a1c9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e89ecef503384f98958d1067d5b7a1c9" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 37493 } } }
I20260923 15:05:43.419163 882 sys_catalog.cc:455] T 00000000000000000000000000000000 P e89ecef503384f98958d1067d5b7a1c9 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e89ecef503384f98958d1067d5b7a1c9. Latest consensus state: current_term: 1 leader_uuid: "e89ecef503384f98958d1067d5b7a1c9" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e89ecef503384f98958d1067d5b7a1c9" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 37493 } } }
I20260923 15:05:43.419209 881 sys_catalog.cc:458] T 00000000000000000000000000000000 P e89ecef503384f98958d1067d5b7a1c9 [sys.catalog]: This master's current role is: LEADER
I20260923 15:05:43.419211 882 sys_catalog.cc:458] T 00000000000000000000000000000000 P e89ecef503384f98958d1067d5b7a1c9 [sys.catalog]: This master's current role is: LEADER
I20260923 15:05:43.420202 889 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 15:05:43.420755 889 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 15:05:43.422432 889 catalog_manager.cc:1347] Generated new cluster ID: 1c863d47723544978c1382af99531d18
I20260923 15:05:43.422487 889 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 15:05:43.430969 889 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 15:05:43.431753 889 catalog_manager.cc:1504] Loading token signing keys...
I20260923 15:05:43.437002 889 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e89ecef503384f98958d1067d5b7a1c9: Generated new TSK 0
I20260923 15:05:43.437260 889 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 15:05:43.444538 13614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKC4soy/build/release/bin/kudu
/tmp/dist-test-taskKC4soy/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790175691491353-13614-0/minicluster-data/ts-0/wal
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790175691491353-13614-0/minicluster-data/ts-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790175691491353-13614-0/minicluster-data/ts-0/logs
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790175691491353-13614-0/minicluster-data/ts-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.75.129:0
--local_ip_for_outbound_sockets=127.13.75.129
--webserver_interface=127.13.75.129
--webserver_port=0
--tserver_master_addrs=127.13.75.190:37493
--builtin_ntp_servers=127.13.75.148:41433
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
W20260923 15:05:43.528115 899 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:05:43.528373 899 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:05:43.528414 899 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:05:43.529691 899 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:05:43.529748 899 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:05:43.529790 899 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.75.129
I20260923 15:05:43.531234 899 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.75.148:41433
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790175691491353-13614-0/minicluster-data/ts-0/data
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790175691491353-13614-0/minicluster-data/ts-0/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.75.129:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790175691491353-13614-0/minicluster-data/ts-0/data/info.pb
--webserver_interface=127.13.75.129
--webserver_port=0
--tserver_master_addrs=127.13.75.190:37493
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.899
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.75.129
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790175691491353-13614-0/minicluster-data/ts-0/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:05:43.531508 899 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:05:43.531833 899 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:05:43.537902 905 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:05:43.537902 907 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:05:43.538012 904 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:05:43.538379 899 server_base.cc:1034] running on GCE node
I20260923 15:05:43.538614 899 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:05:43.538913 899 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:05:43.540100 899 hybrid_clock.cc:648] HybridClock initialized: now 1790175943540072 us; error 43 us; skew 500 ppm
I20260923 15:05:43.541661 899 webserver.cc:460] Webserver started at http://127.13.75.129:38885/ using document root <none> and password file <none>
I20260923 15:05:43.541944 899 fs_manager.cc:359] Metadata directory not provided
I20260923 15:05:43.541991 899 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:05:43.542130 899 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:05:43.543393 899 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790175691491353-13614-0/minicluster-data/ts-0/data/instance:
uuid: "82765766eda54d90b4657a8d9b80c88c"
format_stamp: "Formatted at 2026-09-23 15:05:43 on dist-test-slave-cnb0"
server_key: "cd890ff2d5ab3b11e5b6492d3ec8bbf5"
server_key_iv: "a31d6dc64bc253578c58839bab446aa0"
server_key_version: "encryptionkey@0"
I20260923 15:05:43.543767 899 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790175691491353-13614-0/minicluster-data/ts-0/wal/instance:
uuid: "82765766eda54d90b4657a8d9b80c88c"
format_stamp: "Formatted at 2026-09-23 15:05:43 on dist-test-slave-cnb0"
server_key: "cd890ff2d5ab3b11e5b6492d3ec8bbf5"
server_key_iv: "a31d6dc64bc253578c58839bab446aa0"
server_key_version: "encryptionkey@0"
I20260923 15:05:43.545149 899 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 15:05:43.545938 913 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:05:43.546083 899 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.001s sys 0.000s
I20260923 15:05:43.546202 899 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790175691491353-13614-0/minicluster-data/ts-0/data,/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790175691491353-13614-0/minicluster-data/ts-0/wal
uuid: "82765766eda54d90b4657a8d9b80c88c"
format_stamp: "Formatted at 2026-09-23 15:05:43 on dist-test-slave-cnb0"
server_key: "cd890ff2d5ab3b11e5b6492d3ec8bbf5"
server_key_iv: "a31d6dc64bc253578c58839bab446aa0"
server_key_version: "encryptionkey@0"
I20260923 15:05:43.546279 899 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790175691491353-13614-0/minicluster-data/ts-0/wal
metadata directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790175691491353-13614-0/minicluster-data/ts-0/wal
1 data directories: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790175691491353-13614-0/minicluster-data/ts-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:05:43.557734 899 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:05:43.558048 899 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:05:43.558195 899 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:05:43.558427 899 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 15:05:43.558756 899 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 15:05:43.558789 899 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 15:05:43.558811 899 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 15:05:43.558853 899 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 15:05:43.564914 899 rpc_server.cc:307] RPC server started. Bound to: 127.13.75.129:35957
I20260923 15:05:43.564940 1026 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.75.129:35957 every 8 connection(s)
I20260923 15:05:43.565346 899 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790175691491353-13614-0/minicluster-data/ts-0/data/info.pb
I20260923 15:05:43.570272 13614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKC4soy/build/release/bin/kudu as pid 899
I20260923 15:05:43.570402 13614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790175691491353-13614-0/minicluster-data/ts-0/wal/instance
I20260923 15:05:43.570621 13614 external_mini_cluster.cc:1428] Setting key e7a325d8ff81113bcf9c630714e291df
I20260923 15:05:43.571478 878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:43.566168 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.129:37505 (local address 127.13.75.190:37493)
0923 15:05:43.566229 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0923 15:05:43.566231 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:05:43.566499 (+ 268us) server_negotiation.cc:407] Connection header received
0923 15:05:43.567195 (+ 696us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:05:43.567197 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:05:43.567213 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:05:43.567274 (+ 61us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:05:43.568013 (+ 739us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:05:43.568561 (+ 548us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:05:43.569520 (+ 959us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:05:43.569744 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:05:43.569804 (+ 60us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:05:43.571028 (+ 1224us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:05:43.571030 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:05:43.571030 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 15:05:43.571036 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:05:43.571057 (+ 21us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 15:05:43.571063 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:05:43.571196 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0923 15:05:43.571344 (+ 148us) server_negotiation.cc:299] Negotiation successful
0923 15:05:43.571378 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":23}
I20260923 15:05:43.571669 1029 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:43.566148 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:37493 (local address 127.13.75.129:37505)
0923 15:05:43.566332 (+ 184us) negotiation.cc:107] Waiting for socket to connect
0923 15:05:43.566375 (+ 43us) client_negotiation.cc:174] Beginning negotiation
0923 15:05:43.566999 (+ 624us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:05:43.567333 (+ 334us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:05:43.567336 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:05:43.567344 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:05:43.567921 (+ 577us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:05:43.567926 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:05:43.568644 (+ 718us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:05:43.568645 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:05:43.569434 (+ 789us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:05:43.569436 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:05:43.569830 (+ 394us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:05:43.569831 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:05:43.569944 (+ 113us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:05:43.570763 (+ 819us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 15:05:43.570764 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:05:43.570814 (+ 50us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 15:05:43.570845 (+ 31us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:05:43.571222 (+ 377us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:05:43.571229 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:05:43.571232 (+ 3us) client_negotiation.cc:769] Sending connection context
0923 15:05:43.571309 (+ 77us) client_negotiation.cc:240] Negotiation successful
0923 15:05:43.571425 (+ 116us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119,"thread_start_us":91,"threads_started":1}
I20260923 15:05:43.571861 13614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKC4soy/build/release/bin/kudu
/tmp/dist-test-taskKC4soy/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790175691491353-13614-0/minicluster-data/ts-1/wal
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790175691491353-13614-0/minicluster-data/ts-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790175691491353-13614-0/minicluster-data/ts-1/logs
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790175691491353-13614-0/minicluster-data/ts-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.75.130:0
--local_ip_for_outbound_sockets=127.13.75.130
--webserver_interface=127.13.75.130
--webserver_port=0
--tserver_master_addrs=127.13.75.190:37493
--builtin_ntp_servers=127.13.75.148:41433
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260923 15:05:43.572178 1027 heartbeater.cc:347] Connected to a master server at 127.13.75.190:37493
I20260923 15:05:43.572304 1027 heartbeater.cc:464] Registering TS with master...
I20260923 15:05:43.572506 1027 heartbeater.cc:511] Master 127.13.75.190:37493 requested a full tablet report, sending...
I20260923 15:05:43.573076 839 ts_manager.cc:194] Registered new tserver with Master: 82765766eda54d90b4657a8d9b80c88c (127.13.75.129:35957)
I20260923 15:05:43.573801 839 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.13.75.129:37505
W20260923 15:05:43.655747 1030 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:05:43.655972 1030 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:05:43.655994 1030 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:05:43.657284 1030 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:05:43.657326 1030 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:05:43.657354 1030 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.75.130
I20260923 15:05:43.658736 1030 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.75.148:41433
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790175691491353-13614-0/minicluster-data/ts-1/data
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790175691491353-13614-0/minicluster-data/ts-1/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.75.130:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790175691491353-13614-0/minicluster-data/ts-1/data/info.pb
--webserver_interface=127.13.75.130
--webserver_port=0
--tserver_master_addrs=127.13.75.190:37493
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1030
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.75.130
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790175691491353-13614-0/minicluster-data/ts-1/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:05:43.659031 1030 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:05:43.659409 1030 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:05:43.664526 1036 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:05:43.664579 1038 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:05:43.664526 1035 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:05:43.665469 1030 server_base.cc:1034] running on GCE node
I20260923 15:05:43.665688 1030 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:05:43.665977 1030 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:05:43.667137 1030 hybrid_clock.cc:648] HybridClock initialized: now 1790175943667114 us; error 37 us; skew 500 ppm
I20260923 15:05:43.672082 1030 webserver.cc:460] Webserver started at http://127.13.75.130:38533/ using document root <none> and password file <none>
I20260923 15:05:43.672530 1030 fs_manager.cc:359] Metadata directory not provided
I20260923 15:05:43.672623 1030 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:05:43.672825 1030 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:05:43.674484 1030 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790175691491353-13614-0/minicluster-data/ts-1/data/instance:
uuid: "881282989b014e02af3db0131adc438e"
format_stamp: "Formatted at 2026-09-23 15:05:43 on dist-test-slave-cnb0"
server_key: "8c9008dc51d2b6ea68d0bdf7993de4c5"
server_key_iv: "d3081be909617dd5eeb91fa764be6883"
server_key_version: "encryptionkey@0"
I20260923 15:05:43.674978 1030 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790175691491353-13614-0/minicluster-data/ts-1/wal/instance:
uuid: "881282989b014e02af3db0131adc438e"
format_stamp: "Formatted at 2026-09-23 15:05:43 on dist-test-slave-cnb0"
server_key: "8c9008dc51d2b6ea68d0bdf7993de4c5"
server_key_iv: "d3081be909617dd5eeb91fa764be6883"
server_key_version: "encryptionkey@0"
I20260923 15:05:43.676931 1030 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.000s sys 0.002s
I20260923 15:05:43.677814 1044 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:05:43.678030 1030 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260923 15:05:43.678105 1030 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790175691491353-13614-0/minicluster-data/ts-1/data,/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790175691491353-13614-0/minicluster-data/ts-1/wal
uuid: "881282989b014e02af3db0131adc438e"
format_stamp: "Formatted at 2026-09-23 15:05:43 on dist-test-slave-cnb0"
server_key: "8c9008dc51d2b6ea68d0bdf7993de4c5"
server_key_iv: "d3081be909617dd5eeb91fa764be6883"
server_key_version: "encryptionkey@0"
I20260923 15:05:43.678157 1030 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790175691491353-13614-0/minicluster-data/ts-1/wal
metadata directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790175691491353-13614-0/minicluster-data/ts-1/wal
1 data directories: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790175691491353-13614-0/minicluster-data/ts-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:05:43.697185 1030 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:05:43.697546 1030 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:05:43.697708 1030 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:05:43.697955 1030 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 15:05:43.698325 1030 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 15:05:43.698361 1030 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 15:05:43.698385 1030 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 15:05:43.698398 1030 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 15:05:43.704704 1030 rpc_server.cc:307] RPC server started. Bound to: 127.13.75.130:44339
I20260923 15:05:43.704756 1158 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.75.130:44339 every 8 connection(s)
I20260923 15:05:43.705103 1030 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790175691491353-13614-0/minicluster-data/ts-1/data/info.pb
I20260923 15:05:43.707440 13614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKC4soy/build/release/bin/kudu as pid 1030
I20260923 15:05:43.707558 13614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790175691491353-13614-0/minicluster-data/ts-1/wal/instance
I20260923 15:05:43.707798 13614 external_mini_cluster.cc:1428] Setting key a6ba22f67bf89cc042fa97ddb317ceef
I20260923 15:05:43.708841 13614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKC4soy/build/release/bin/kudu
/tmp/dist-test-taskKC4soy/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790175691491353-13614-0/minicluster-data/ts-2/wal
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790175691491353-13614-0/minicluster-data/ts-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790175691491353-13614-0/minicluster-data/ts-2/logs
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790175691491353-13614-0/minicluster-data/ts-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--encrypt_data_at_rest=true
--ipki_server_key_size=768
--openssl_security_level_override=0
tserver
run
--rpc_bind_addresses=127.13.75.131:0
--local_ip_for_outbound_sockets=127.13.75.131
--webserver_interface=127.13.75.131
--webserver_port=0
--tserver_master_addrs=127.13.75.190:37493
--builtin_ntp_servers=127.13.75.148:41433
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_trace_negotiation
--rpc_authentication=disabled with env {}
I20260923 15:05:43.712008 878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:43.705922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.130:43129 (local address 127.13.75.190:37493)
0923 15:05:43.706005 (+ 83us) server_negotiation.cc:206] Beginning negotiation
0923 15:05:43.706008 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:05:43.706310 (+ 302us) server_negotiation.cc:407] Connection header received
0923 15:05:43.706905 (+ 595us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:05:43.706906 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:05:43.706919 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:05:43.706996 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:05:43.708032 (+ 1036us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:05:43.708688 (+ 656us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:05:43.710227 (+ 1539us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:05:43.710579 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:05:43.710667 (+ 88us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:05:43.711699 (+ 1032us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:05:43.711700 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:05:43.711701 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 15:05:43.711706 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:05:43.711724 (+ 18us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 15:05:43.711729 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:05:43.711788 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0923 15:05:43.711861 (+ 73us) server_negotiation.cc:299] Negotiation successful
0923 15:05:43.711888 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":49}
I20260923 15:05:43.712172 1161 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:43.705893 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:37493 (local address 127.13.75.130:43129)
0923 15:05:43.706169 (+ 276us) negotiation.cc:107] Waiting for socket to connect
0923 15:05:43.706209 (+ 40us) client_negotiation.cc:174] Beginning negotiation
0923 15:05:43.706743 (+ 534us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:05:43.707139 (+ 396us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:05:43.707143 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:05:43.707151 (+ 8us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:05:43.707914 (+ 763us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:05:43.707921 (+ 7us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:05:43.708787 (+ 866us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:05:43.708788 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:05:43.710118 (+ 1330us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:05:43.710122 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:05:43.710680 (+ 558us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:05:43.710681 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:05:43.710809 (+ 128us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:05:43.711521 (+ 712us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 15:05:43.711522 (+ 1us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:05:43.711553 (+ 31us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 15:05:43.711592 (+ 39us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:05:43.711795 (+ 203us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:05:43.711800 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:05:43.711802 (+ 2us) client_negotiation.cc:769] Sending connection context
0923 15:05:43.711857 (+ 55us) client_negotiation.cc:240] Negotiation successful
0923 15:05:43.711944 (+ 87us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":191,"thread_start_us":119,"threads_started":1}
I20260923 15:05:43.712636 1159 heartbeater.cc:347] Connected to a master server at 127.13.75.190:37493
I20260923 15:05:43.712750 1159 heartbeater.cc:464] Registering TS with master...
I20260923 15:05:43.712986 1159 heartbeater.cc:511] Master 127.13.75.190:37493 requested a full tablet report, sending...
I20260923 15:05:43.713449 839 ts_manager.cc:194] Registered new tserver with Master: 881282989b014e02af3db0131adc438e (127.13.75.130:44339)
I20260923 15:05:43.713827 839 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.13.75.130:43129
W20260923 15:05:43.793900 1162 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:05:43.794124 1162 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:05:43.794147 1162 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:05:43.795413 1162 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:05:43.795459 1162 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:05:43.795487 1162 flags.cc:425] Enabled experimental flag: --local_ip_for_outbound_sockets=127.13.75.131
I20260923 15:05:43.796885 1162 tablet_server_runner.cc:78] Tablet server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.75.148:41433
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790175691491353-13614-0/minicluster-data/ts-2/data
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790175691491353-13614-0/minicluster-data/ts-2/wal
--rpc_trace_negotiation=true
--ipki_server_key_size=768
--openssl_security_level_override=0
--rpc_bind_addresses=127.13.75.131:0
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790175691491353-13614-0/minicluster-data/ts-2/data/info.pb
--webserver_interface=127.13.75.131
--webserver_port=0
--tserver_master_addrs=127.13.75.190:37493
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1162
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--local_ip_for_outbound_sockets=127.13.75.131
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790175691491353-13614-0/minicluster-data/ts-2/logs
--logbuflevel=-1
--logtostderr=true
Tablet server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:05:43.797160 1162 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:05:43.797485 1162 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:05:43.803213 1170 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:05:43.803211 1167 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:05:43.803364 1162 server_base.cc:1034] running on GCE node
W20260923 15:05:43.803329 1168 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:05:43.803755 1162 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:05:43.804042 1162 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:05:43.805209 1162 hybrid_clock.cc:648] HybridClock initialized: now 1790175943805194 us; error 30 us; skew 500 ppm
I20260923 15:05:43.806788 1162 webserver.cc:460] Webserver started at http://127.13.75.131:46199/ using document root <none> and password file <none>
I20260923 15:05:43.807088 1162 fs_manager.cc:359] Metadata directory not provided
I20260923 15:05:43.807157 1162 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:05:43.807264 1162 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:05:43.808421 1162 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790175691491353-13614-0/minicluster-data/ts-2/data/instance:
uuid: "c64be363c6354f7d915f4a4bd93828af"
format_stamp: "Formatted at 2026-09-23 15:05:43 on dist-test-slave-cnb0"
server_key: "70cb03fae9c6979d2ec6083c8cf184ab"
server_key_iv: "cc80447181ecd2f4ec9bceb8b1672947"
server_key_version: "encryptionkey@0"
I20260923 15:05:43.808761 1162 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790175691491353-13614-0/minicluster-data/ts-2/wal/instance:
uuid: "c64be363c6354f7d915f4a4bd93828af"
format_stamp: "Formatted at 2026-09-23 15:05:43 on dist-test-slave-cnb0"
server_key: "70cb03fae9c6979d2ec6083c8cf184ab"
server_key_iv: "cc80447181ecd2f4ec9bceb8b1672947"
server_key_version: "encryptionkey@0"
I20260923 15:05:43.810149 1162 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 15:05:43.810961 1176 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:05:43.811148 1162 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:05:43.811247 1162 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790175691491353-13614-0/minicluster-data/ts-2/data,/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790175691491353-13614-0/minicluster-data/ts-2/wal
uuid: "c64be363c6354f7d915f4a4bd93828af"
format_stamp: "Formatted at 2026-09-23 15:05:43 on dist-test-slave-cnb0"
server_key: "70cb03fae9c6979d2ec6083c8cf184ab"
server_key_iv: "cc80447181ecd2f4ec9bceb8b1672947"
server_key_version: "encryptionkey@0"
I20260923 15:05:43.811311 1162 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790175691491353-13614-0/minicluster-data/ts-2/wal
metadata directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790175691491353-13614-0/minicluster-data/ts-2/wal
1 data directories: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790175691491353-13614-0/minicluster-data/ts-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:05:43.819324 1162 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:05:43.819681 1162 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:05:43.819795 1162 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:05:43.820025 1162 txn_system_client.cc:433] TxnSystemClient initialization is disabled...
I20260923 15:05:43.820369 1162 ts_tablet_manager.cc:579] Loaded tablet metadata (0 total tablets, 0 live tablets)
I20260923 15:05:43.820405 1162 ts_tablet_manager.cc:525] Time spent load tablet metadata: real 0.000s user 0.000s sys 0.000s
I20260923 15:05:43.820427 1162 ts_tablet_manager.cc:610] Registered 0 tablets
I20260923 15:05:43.820439 1162 ts_tablet_manager.cc:589] Time spent register tablets: real 0.000s user 0.000s sys 0.000s
I20260923 15:05:43.826040 1162 rpc_server.cc:307] RPC server started. Bound to: 127.13.75.131:41577
I20260923 15:05:43.826118 1289 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.75.131:41577 every 8 connection(s)
I20260923 15:05:43.826472 1162 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790175691491353-13614-0/minicluster-data/ts-2/data/info.pb
I20260923 15:05:43.832265 878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:43.827305 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.131:45897 (local address 127.13.75.190:37493)
0923 15:05:43.827426 (+ 121us) server_negotiation.cc:206] Beginning negotiation
0923 15:05:43.827429 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:05:43.827628 (+ 199us) server_negotiation.cc:407] Connection header received
0923 15:05:43.828182 (+ 554us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:05:43.828183 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:05:43.828203 (+ 20us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:05:43.828274 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:05:43.829033 (+ 759us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:05:43.829571 (+ 538us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:05:43.830603 (+ 1032us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:05:43.830816 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:05:43.830916 (+ 100us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:05:43.831908 (+ 992us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:05:43.831909 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:05:43.831910 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 15:05:43.831915 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:05:43.831931 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 15:05:43.831941 (+ 10us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:05:43.832003 (+ 62us) server_negotiation.cc:1035] Waiting for connection context
0923 15:05:43.832098 (+ 95us) server_negotiation.cc:299] Negotiation successful
0923 15:05:43.832128 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86}
I20260923 15:05:43.832479 1292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:43.827227 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:37493 (local address 127.13.75.131:45897)
0923 15:05:43.827481 (+ 254us) negotiation.cc:107] Waiting for socket to connect
0923 15:05:43.827532 (+ 51us) client_negotiation.cc:174] Beginning negotiation
0923 15:05:43.828071 (+ 539us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:05:43.828351 (+ 280us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:05:43.828355 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:05:43.828365 (+ 10us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:05:43.828931 (+ 566us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:05:43.828936 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:05:43.829675 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:05:43.829676 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:05:43.830507 (+ 831us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:05:43.830509 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:05:43.830967 (+ 458us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:05:43.830968 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:05:43.831099 (+ 131us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:05:43.831749 (+ 650us) client_negotiation.cc:623] Initiating SASL PLAIN handshake
0923 15:05:43.831749 (+ 0us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:05:43.831778 (+ 29us) client_negotiation.cc:814] callback for SASL_CB_AUTHNAME
0923 15:05:43.831802 (+ 24us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:05:43.832003 (+ 201us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:05:43.832009 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:05:43.832013 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 15:05:43.832102 (+ 89us) client_negotiation.cc:240] Negotiation successful
0923 15:05:43.832213 (+ 111us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":160,"thread_start_us":87,"threads_started":1}
I20260923 15:05:43.833004 1290 heartbeater.cc:347] Connected to a master server at 127.13.75.190:37493
I20260923 15:05:43.833104 1290 heartbeater.cc:464] Registering TS with master...
I20260923 15:05:43.833298 1290 heartbeater.cc:511] Master 127.13.75.190:37493 requested a full tablet report, sending...
I20260923 15:05:43.833721 839 ts_manager.cc:194] Registered new tserver with Master: c64be363c6354f7d915f4a4bd93828af (127.13.75.131:41577)
I20260923 15:05:43.834081 839 master_service.cc:496] Signed X509 certificate for tserver {username='slave'} at 127.13.75.131:45897
I20260923 15:05:43.835012 13614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKC4soy/build/release/bin/kudu as pid 1162
I20260923 15:05:43.835100 13614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790175691491353-13614-0/minicluster-data/ts-2/wal/instance
I20260923 15:05:43.835276 13614 external_mini_cluster.cc:1428] Setting key 5ae129d0c3ecbdb704ec2216a6dbae81
I20260923 15:05:43.836476 13614 external_mini_cluster.cc:930] 3 TS(s) registered with all masters
I20260923 15:05:43.841097 878 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:43.838000 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45958 (local address 127.13.75.190:37493)
0923 15:05:43.838081 (+ 81us) server_negotiation.cc:206] Beginning negotiation
0923 15:05:43.838084 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:05:43.838178 (+ 94us) server_negotiation.cc:407] Connection header received
0923 15:05:43.838312 (+ 134us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:05:43.838314 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:05:43.838324 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:05:43.838381 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:05:43.838552 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:05:43.839129 (+ 577us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:05:43.839902 (+ 773us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:05:43.840091 (+ 189us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:05:43.840194 (+ 103us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:05:43.840837 (+ 643us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:05:43.840839 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:05:43.840839 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 15:05:43.840844 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:05:43.840858 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 15:05:43.840861 (+ 3us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:05:43.840912 (+ 51us) server_negotiation.cc:1035] Waiting for connection context
0923 15:05:43.840985 (+ 73us) server_negotiation.cc:299] Negotiation successful
0923 15:05:43.841008 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":53}
I20260923 15:05:43.842748 839 catalog_manager.cc:1908] Servicing CreateTable request from {username='slave'} at 127.0.0.1:45958:
name: "test-table"
schema {
columns {
name: "key"
type: INT32
is_key: true
is_nullable: false
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
columns {
name: "val"
type: INT32
is_key: false
is_nullable: true
encoding: AUTO_ENCODING
compression: DEFAULT_COMPRESSION
cfile_block_size: 0
immutable: false
}
}
num_replicas: 3
split_rows_range_bounds {
}
partition_schema {
range_schema {
columns {
name: "key"
}
}
}
W20260923 15:05:43.844372 839 catalog_manager.cc:6854] The number of live tablet servers is not enough to re-replicate a tablet replica of the newly created table test-table in case of a server failure: 4 tablet servers would be needed, 3 are available. Consider bringing up more tablet servers.
I20260923 15:05:43.851385 1303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:43.846974 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40296 (local address 127.13.75.130:44339)
0923 15:05:43.847784 (+ 810us) server_negotiation.cc:206] Beginning negotiation
0923 15:05:43.847789 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 15:05:43.847798 (+ 9us) server_negotiation.cc:407] Connection header received
0923 15:05:43.847840 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:05:43.847842 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:05:43.847856 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:05:43.847933 (+ 77us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:05:43.848030 (+ 97us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:05:43.849073 (+ 1043us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:05:43.850075 (+ 1002us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:05:43.850867 (+ 792us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:05:43.851007 (+ 140us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:05:43.851115 (+ 108us) server_negotiation.cc:1035] Waiting for connection context
0923 15:05:43.851241 (+ 126us) server_negotiation.cc:299] Negotiation successful
0923 15:05:43.851285 (+ 44us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":769,"thread_start_us":89,"threads_started":1}
I20260923 15:05:43.851408 1298 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:43.846674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.130:44339 (local address 127.0.0.1:40296)
0923 15:05:43.847143 (+ 469us) negotiation.cc:107] Waiting for socket to connect
0923 15:05:43.847153 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 15:05:43.847191 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:05:43.847932 (+ 741us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:05:43.847933 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:05:43.847935 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:05:43.847969 (+ 34us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:05:43.847970 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:05:43.849159 (+ 1189us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:05:43.849160 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:05:43.849977 (+ 817us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:05:43.849979 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:05:43.851027 (+ 1048us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:05:43.851028 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:05:43.851123 (+ 95us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:05:43.851140 (+ 17us) client_negotiation.cc:769] Sending connection context
0923 15:05:43.851221 (+ 81us) client_negotiation.cc:240] Negotiation successful
0923 15:05:43.851282 (+ 61us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":457,"mutex_wait_us":2,"thread_start_us":92,"threads_started":1}
I20260923 15:05:43.851538 1302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:43.846933 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49566 (local address 127.13.75.129:35957)
0923 15:05:43.847477 (+ 544us) server_negotiation.cc:206] Beginning negotiation
0923 15:05:43.847480 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:05:43.847509 (+ 29us) server_negotiation.cc:407] Connection header received
0923 15:05:43.847551 (+ 42us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:05:43.847553 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:05:43.847569 (+ 16us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:05:43.847645 (+ 76us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:05:43.847827 (+ 182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:05:43.848698 (+ 871us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:05:43.850464 (+ 1766us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:05:43.851061 (+ 597us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:05:43.851129 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:05:43.851221 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0923 15:05:43.851400 (+ 179us) server_negotiation.cc:299] Negotiation successful
0923 15:05:43.851449 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":482,"thread_start_us":87,"threads_started":1}
I20260923 15:05:43.851734 1299 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:43.846686 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.129:35957 (local address 127.0.0.1:49566)
0923 15:05:43.847263 (+ 577us) negotiation.cc:107] Waiting for socket to connect
0923 15:05:43.847272 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 15:05:43.847310 (+ 38us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:05:43.847652 (+ 342us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:05:43.847653 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:05:43.847655 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:05:43.847693 (+ 38us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:05:43.847694 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:05:43.848811 (+ 1117us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:05:43.848812 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:05:43.850385 (+ 1573us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:05:43.850387 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:05:43.851256 (+ 869us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:05:43.851257 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:05:43.851307 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:05:43.851311 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 15:05:43.851636 (+ 325us) client_negotiation.cc:240] Negotiation successful
0923 15:05:43.851683 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":564,"thread_start_us":127,"threads_started":1}
I20260923 15:05:43.851994 1300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:43.846674 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.131:41577 (local address 127.0.0.1:53104)
0923 15:05:43.846918 (+ 244us) negotiation.cc:107] Waiting for socket to connect
0923 15:05:43.846962 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0923 15:05:43.847032 (+ 70us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:05:43.847369 (+ 337us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:05:43.847372 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:05:43.847375 (+ 3us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:05:43.847434 (+ 59us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:05:43.847436 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:05:43.849023 (+ 1587us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:05:43.849024 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:05:43.850154 (+ 1130us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:05:43.850157 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:05:43.851178 (+ 1021us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:05:43.851179 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:05:43.851269 (+ 90us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:05:43.851273 (+ 4us) client_negotiation.cc:769] Sending connection context
0923 15:05:43.851328 (+ 55us) client_negotiation.cc:240] Negotiation successful
0923 15:05:43.851369 (+ 41us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":193,"mutex_wait_us":41,"spinlock_wait_cycles":896,"thread_start_us":129,"threads_started":1}
I20260923 15:05:43.852305 1301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:43.846919 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53104 (local address 127.13.75.131:41577)
0923 15:05:43.847129 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0923 15:05:43.847134 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 15:05:43.847145 (+ 11us) server_negotiation.cc:407] Connection header received
0923 15:05:43.847191 (+ 46us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:05:43.847194 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:05:43.847211 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:05:43.847347 (+ 136us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:05:43.847624 (+ 277us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:05:43.848937 (+ 1313us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:05:43.850275 (+ 1338us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:05:43.851007 (+ 732us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:05:43.851064 (+ 57us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:05:43.851136 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0923 15:05:43.852157 (+ 1021us) server_negotiation.cc:299] Negotiation successful
0923 15:05:43.852203 (+ 46us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":85,"threads_started":1}
I20260923 15:05:43.852332 1092 tablet_service.cc:1467] Processing CreateTablet for tablet e6c828d6eac6488d9533a7d71561f7e0 (DEFAULT_TABLE table=test-table [id=910e201d37c24026b18dafe8af9fadf6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 15:05:43.852403 961 tablet_service.cc:1467] Processing CreateTablet for tablet e6c828d6eac6488d9533a7d71561f7e0 (DEFAULT_TABLE table=test-table [id=910e201d37c24026b18dafe8af9fadf6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 15:05:43.852731 1092 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e6c828d6eac6488d9533a7d71561f7e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:05:43.852741 961 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e6c828d6eac6488d9533a7d71561f7e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:05:43.852964 1224 tablet_service.cc:1467] Processing CreateTablet for tablet e6c828d6eac6488d9533a7d71561f7e0 (DEFAULT_TABLE table=test-table [id=910e201d37c24026b18dafe8af9fadf6]), partition=RANGE (key) PARTITION UNBOUNDED
I20260923 15:05:43.853278 1224 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet e6c828d6eac6488d9533a7d71561f7e0. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:05:43.856052 1304 tablet_bootstrap.cc:492] T e6c828d6eac6488d9533a7d71561f7e0 P c64be363c6354f7d915f4a4bd93828af: Bootstrap starting.
I20260923 15:05:43.856216 1305 tablet_bootstrap.cc:492] T e6c828d6eac6488d9533a7d71561f7e0 P 82765766eda54d90b4657a8d9b80c88c: Bootstrap starting.
I20260923 15:05:43.856820 1304 tablet_bootstrap.cc:654] T e6c828d6eac6488d9533a7d71561f7e0 P c64be363c6354f7d915f4a4bd93828af: Neither blocks nor log segments found. Creating new log.
I20260923 15:05:43.856833 1305 tablet_bootstrap.cc:654] T e6c828d6eac6488d9533a7d71561f7e0 P 82765766eda54d90b4657a8d9b80c88c: Neither blocks nor log segments found. Creating new log.
I20260923 15:05:43.857136 1304 log.cc:824] T e6c828d6eac6488d9533a7d71561f7e0 P c64be363c6354f7d915f4a4bd93828af: Log is configured to *not* fsync() on all Append() calls
I20260923 15:05:43.857141 1305 log.cc:824] T e6c828d6eac6488d9533a7d71561f7e0 P 82765766eda54d90b4657a8d9b80c88c: Log is configured to *not* fsync() on all Append() calls
I20260923 15:05:43.857494 1306 tablet_bootstrap.cc:492] T e6c828d6eac6488d9533a7d71561f7e0 P 881282989b014e02af3db0131adc438e: Bootstrap starting.
I20260923 15:05:43.857961 1305 tablet_bootstrap.cc:492] T e6c828d6eac6488d9533a7d71561f7e0 P 82765766eda54d90b4657a8d9b80c88c: No bootstrap required, opened a new log
I20260923 15:05:43.858036 1305 ts_tablet_manager.cc:1397] T e6c828d6eac6488d9533a7d71561f7e0 P 82765766eda54d90b4657a8d9b80c88c: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 15:05:43.858194 1304 tablet_bootstrap.cc:492] T e6c828d6eac6488d9533a7d71561f7e0 P c64be363c6354f7d915f4a4bd93828af: No bootstrap required, opened a new log
I20260923 15:05:43.858217 1306 tablet_bootstrap.cc:654] T e6c828d6eac6488d9533a7d71561f7e0 P 881282989b014e02af3db0131adc438e: Neither blocks nor log segments found. Creating new log.
I20260923 15:05:43.858294 1304 ts_tablet_manager.cc:1397] T e6c828d6eac6488d9533a7d71561f7e0 P c64be363c6354f7d915f4a4bd93828af: Time spent bootstrapping tablet: real 0.002s user 0.002s sys 0.000s
I20260923 15:05:43.858548 1306 log.cc:824] T e6c828d6eac6488d9533a7d71561f7e0 P 881282989b014e02af3db0131adc438e: Log is configured to *not* fsync() on all Append() calls
I20260923 15:05:43.859421 1306 tablet_bootstrap.cc:492] T e6c828d6eac6488d9533a7d71561f7e0 P 881282989b014e02af3db0131adc438e: No bootstrap required, opened a new log
I20260923 15:05:43.859493 1306 ts_tablet_manager.cc:1397] T e6c828d6eac6488d9533a7d71561f7e0 P 881282989b014e02af3db0131adc438e: Time spent bootstrapping tablet: real 0.002s user 0.001s sys 0.000s
I20260923 15:05:43.860424 1304 raft_consensus.cc:348] T e6c828d6eac6488d9533a7d71561f7e0 P c64be363c6354f7d915f4a4bd93828af [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c64be363c6354f7d915f4a4bd93828af" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 41577 } } peers { permanent_uuid: "881282989b014e02af3db0131adc438e" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 44339 } } peers { permanent_uuid: "82765766eda54d90b4657a8d9b80c88c" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 35957 } }
I20260923 15:05:43.860582 1304 raft_consensus.cc:374] T e6c828d6eac6488d9533a7d71561f7e0 P c64be363c6354f7d915f4a4bd93828af [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:05:43.860627 1304 raft_consensus.cc:729] T e6c828d6eac6488d9533a7d71561f7e0 P c64be363c6354f7d915f4a4bd93828af [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c64be363c6354f7d915f4a4bd93828af, State: Initialized, Role: FOLLOWER
I20260923 15:05:43.860757 1304 consensus_queue.cc:260] T e6c828d6eac6488d9533a7d71561f7e0 P c64be363c6354f7d915f4a4bd93828af [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c64be363c6354f7d915f4a4bd93828af" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 41577 } } peers { permanent_uuid: "881282989b014e02af3db0131adc438e" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 44339 } } peers { permanent_uuid: "82765766eda54d90b4657a8d9b80c88c" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 35957 } }
I20260923 15:05:43.860824 1305 raft_consensus.cc:348] T e6c828d6eac6488d9533a7d71561f7e0 P 82765766eda54d90b4657a8d9b80c88c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c64be363c6354f7d915f4a4bd93828af" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 41577 } } peers { permanent_uuid: "881282989b014e02af3db0131adc438e" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 44339 } } peers { permanent_uuid: "82765766eda54d90b4657a8d9b80c88c" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 35957 } }
I20260923 15:05:43.860973 1305 raft_consensus.cc:374] T e6c828d6eac6488d9533a7d71561f7e0 P 82765766eda54d90b4657a8d9b80c88c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:05:43.861006 1305 raft_consensus.cc:729] T e6c828d6eac6488d9533a7d71561f7e0 P 82765766eda54d90b4657a8d9b80c88c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 82765766eda54d90b4657a8d9b80c88c, State: Initialized, Role: FOLLOWER
I20260923 15:05:43.861045 1304 ts_tablet_manager.cc:1428] T e6c828d6eac6488d9533a7d71561f7e0 P c64be363c6354f7d915f4a4bd93828af: Time spent starting tablet: real 0.003s user 0.003s sys 0.000s
I20260923 15:05:43.861160 1290 heartbeater.cc:503] Master 127.13.75.190:37493 was elected leader, sending a full tablet report...
I20260923 15:05:43.861131 1305 consensus_queue.cc:260] T e6c828d6eac6488d9533a7d71561f7e0 P 82765766eda54d90b4657a8d9b80c88c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c64be363c6354f7d915f4a4bd93828af" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 41577 } } peers { permanent_uuid: "881282989b014e02af3db0131adc438e" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 44339 } } peers { permanent_uuid: "82765766eda54d90b4657a8d9b80c88c" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 35957 } }
I20260923 15:05:43.861387 1305 ts_tablet_manager.cc:1428] T e6c828d6eac6488d9533a7d71561f7e0 P 82765766eda54d90b4657a8d9b80c88c: Time spent starting tablet: real 0.003s user 0.000s sys 0.004s
I20260923 15:05:43.861500 1027 heartbeater.cc:503] Master 127.13.75.190:37493 was elected leader, sending a full tablet report...
I20260923 15:05:43.862130 1306 raft_consensus.cc:348] T e6c828d6eac6488d9533a7d71561f7e0 P 881282989b014e02af3db0131adc438e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c64be363c6354f7d915f4a4bd93828af" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 41577 } } peers { permanent_uuid: "881282989b014e02af3db0131adc438e" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 44339 } } peers { permanent_uuid: "82765766eda54d90b4657a8d9b80c88c" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 35957 } }
I20260923 15:05:43.862288 1306 raft_consensus.cc:374] T e6c828d6eac6488d9533a7d71561f7e0 P 881282989b014e02af3db0131adc438e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:05:43.862335 1306 raft_consensus.cc:729] T e6c828d6eac6488d9533a7d71561f7e0 P 881282989b014e02af3db0131adc438e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 881282989b014e02af3db0131adc438e, State: Initialized, Role: FOLLOWER
I20260923 15:05:43.862478 1306 consensus_queue.cc:260] T e6c828d6eac6488d9533a7d71561f7e0 P 881282989b014e02af3db0131adc438e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c64be363c6354f7d915f4a4bd93828af" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 41577 } } peers { permanent_uuid: "881282989b014e02af3db0131adc438e" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 44339 } } peers { permanent_uuid: "82765766eda54d90b4657a8d9b80c88c" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 35957 } }
I20260923 15:05:43.862764 1306 ts_tablet_manager.cc:1428] T e6c828d6eac6488d9533a7d71561f7e0 P 881282989b014e02af3db0131adc438e: Time spent starting tablet: real 0.003s user 0.004s sys 0.000s
I20260923 15:05:43.862830 1159 heartbeater.cc:503] Master 127.13.75.190:37493 was elected leader, sending a full tablet report...
I20260923 15:05:43.949368 1310 raft_consensus.cc:482] T e6c828d6eac6488d9533a7d71561f7e0 P c64be363c6354f7d915f4a4bd93828af [term 0 FOLLOWER]: Starting pre-election (no leader contacted us within the election timeout)
I20260923 15:05:43.949542 1310 raft_consensus.cc:504] T e6c828d6eac6488d9533a7d71561f7e0 P c64be363c6354f7d915f4a4bd93828af [term 0 FOLLOWER]: Starting pre-election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c64be363c6354f7d915f4a4bd93828af" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 41577 } } peers { permanent_uuid: "881282989b014e02af3db0131adc438e" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 44339 } } peers { permanent_uuid: "82765766eda54d90b4657a8d9b80c88c" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 35957 } }
I20260923 15:05:43.950044 1310 leader_election.cc:288] T e6c828d6eac6488d9533a7d71561f7e0 P c64be363c6354f7d915f4a4bd93828af [CANDIDATE]: Term 1 pre-election: Requested pre-vote from peers 881282989b014e02af3db0131adc438e (127.13.75.130:44339), 82765766eda54d90b4657a8d9b80c88c (127.13.75.129:35957)
I20260923 15:05:43.954558 1292 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:43.950290 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.130:44339 (local address 127.13.75.131:36399)
0923 15:05:43.950404 (+ 114us) negotiation.cc:107] Waiting for socket to connect
0923 15:05:43.950419 (+ 15us) client_negotiation.cc:174] Beginning negotiation
0923 15:05:43.950473 (+ 54us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:05:43.950816 (+ 343us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:05:43.950818 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:05:43.950819 (+ 1us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:05:43.950872 (+ 53us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:05:43.950873 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:05:43.951960 (+ 1087us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:05:43.951962 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:05:43.953424 (+ 1462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:05:43.953427 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:05:43.954256 (+ 829us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:05:43.954257 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:05:43.954342 (+ 85us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:05:43.954347 (+ 5us) client_negotiation.cc:769] Sending connection context
0923 15:05:43.954410 (+ 63us) client_negotiation.cc:240] Negotiation successful
0923 15:05:43.954460 (+ 50us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":93,"mutex_wait_us":143}
I20260923 15:05:43.954581 1303 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:43.950439 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.131:36399 (local address 127.13.75.130:44339)
0923 15:05:43.950505 (+ 66us) server_negotiation.cc:206] Beginning negotiation
0923 15:05:43.950508 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:05:43.950520 (+ 12us) server_negotiation.cc:407] Connection header received
0923 15:05:43.950557 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:05:43.950558 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:05:43.950568 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:05:43.950638 (+ 70us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:05:43.950975 (+ 337us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:05:43.951837 (+ 862us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:05:43.953529 (+ 1692us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:05:43.954149 (+ 620us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:05:43.954234 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:05:43.954245 (+ 11us) server_negotiation.cc:1035] Waiting for connection context
0923 15:05:43.954452 (+ 207us) server_negotiation.cc:299] Negotiation successful
0923 15:05:43.954499 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260923 15:05:43.954581 1302 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:43.950521 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.13.75.131:59373 (local address 127.13.75.129:35957)
0923 15:05:43.950582 (+ 61us) server_negotiation.cc:206] Beginning negotiation
0923 15:05:43.950585 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:05:43.950595 (+ 10us) server_negotiation.cc:407] Connection header received
0923 15:05:43.950652 (+ 57us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:05:43.950653 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:05:43.950691 (+ 38us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:05:43.950757 (+ 66us) server_negotiation.cc:226] Negotiated authn=CERTIFICATE
0923 15:05:43.950978 (+ 221us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:05:43.951836 (+ 858us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:05:43.953534 (+ 1698us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:05:43.954150 (+ 616us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:05:43.954280 (+ 130us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:05:43.954290 (+ 10us) server_negotiation.cc:1035] Waiting for connection context
0923 15:05:43.954456 (+ 166us) server_negotiation.cc:299] Negotiation successful
0923 15:05:43.954499 (+ 43us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":34}
I20260923 15:05:43.954573 1313 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:43.950335 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.129:35957 (local address 127.13.75.131:59373)
0923 15:05:43.950538 (+ 203us) negotiation.cc:107] Waiting for socket to connect
0923 15:05:43.950547 (+ 9us) client_negotiation.cc:174] Beginning negotiation
0923 15:05:43.950592 (+ 45us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:05:43.950761 (+ 169us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:05:43.950762 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:05:43.950764 (+ 2us) client_negotiation.cc:189] Negotiated authn=CERTIFICATE
0923 15:05:43.950856 (+ 92us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:05:43.950858 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:05:43.951960 (+ 1102us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:05:43.951962 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:05:43.953424 (+ 1462us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:05:43.953427 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:05:43.954279 (+ 852us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:05:43.954280 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:05:43.954342 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:05:43.954347 (+ 5us) client_negotiation.cc:769] Sending connection context
0923 15:05:43.954410 (+ 63us) client_negotiation.cc:240] Negotiation successful
0923 15:05:43.954459 (+ 49us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165,"mutex_wait_us":110,"thread_start_us":101,"threads_started":1}
I20260923 15:05:43.954948 1113 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6c828d6eac6488d9533a7d71561f7e0" candidate_uuid: "c64be363c6354f7d915f4a4bd93828af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "881282989b014e02af3db0131adc438e" is_pre_election: true
I20260923 15:05:43.954973 981 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6c828d6eac6488d9533a7d71561f7e0" candidate_uuid: "c64be363c6354f7d915f4a4bd93828af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "82765766eda54d90b4657a8d9b80c88c" is_pre_election: true
I20260923 15:05:43.955117 1113 raft_consensus.cc:2454] T e6c828d6eac6488d9533a7d71561f7e0 P 881282989b014e02af3db0131adc438e [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c64be363c6354f7d915f4a4bd93828af in term 0.
I20260923 15:05:43.955188 981 raft_consensus.cc:2454] T e6c828d6eac6488d9533a7d71561f7e0 P 82765766eda54d90b4657a8d9b80c88c [term 0 FOLLOWER]: Leader pre-election vote request: Granting yes vote for candidate c64be363c6354f7d915f4a4bd93828af in term 0.
I20260923 15:05:43.955381 1177 leader_election.cc:302] T e6c828d6eac6488d9533a7d71561f7e0 P c64be363c6354f7d915f4a4bd93828af [CANDIDATE]: Term 1 pre-election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 881282989b014e02af3db0131adc438e, c64be363c6354f7d915f4a4bd93828af; no voters:
I20260923 15:05:43.955575 1310 raft_consensus.cc:2781] T e6c828d6eac6488d9533a7d71561f7e0 P c64be363c6354f7d915f4a4bd93828af [term 0 FOLLOWER]: Leader pre-election won for term 1
I20260923 15:05:43.955670 1310 raft_consensus.cc:482] T e6c828d6eac6488d9533a7d71561f7e0 P c64be363c6354f7d915f4a4bd93828af [term 0 FOLLOWER]: Starting leader election (no leader contacted us within the election timeout)
W20260923 15:05:43.955675 1160 tablet.cc:2369] T e6c828d6eac6488d9533a7d71561f7e0 P 881282989b014e02af3db0131adc438e: Can't schedule compaction. Clean time has not been advanced past its initial value.
I20260923 15:05:43.955716 1310 raft_consensus.cc:3037] T e6c828d6eac6488d9533a7d71561f7e0 P c64be363c6354f7d915f4a4bd93828af [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:05:43.956502 1310 raft_consensus.cc:504] T e6c828d6eac6488d9533a7d71561f7e0 P c64be363c6354f7d915f4a4bd93828af [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c64be363c6354f7d915f4a4bd93828af" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 41577 } } peers { permanent_uuid: "881282989b014e02af3db0131adc438e" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 44339 } } peers { permanent_uuid: "82765766eda54d90b4657a8d9b80c88c" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 35957 } }
I20260923 15:05:43.956673 1310 leader_election.cc:288] T e6c828d6eac6488d9533a7d71561f7e0 P c64be363c6354f7d915f4a4bd93828af [CANDIDATE]: Term 1 election: Requested vote from peers 881282989b014e02af3db0131adc438e (127.13.75.130:44339), 82765766eda54d90b4657a8d9b80c88c (127.13.75.129:35957)
I20260923 15:05:43.956848 981 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6c828d6eac6488d9533a7d71561f7e0" candidate_uuid: "c64be363c6354f7d915f4a4bd93828af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "82765766eda54d90b4657a8d9b80c88c"
I20260923 15:05:43.956881 1113 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "e6c828d6eac6488d9533a7d71561f7e0" candidate_uuid: "c64be363c6354f7d915f4a4bd93828af" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: false dest_uuid: "881282989b014e02af3db0131adc438e"
I20260923 15:05:43.956950 1113 raft_consensus.cc:3037] T e6c828d6eac6488d9533a7d71561f7e0 P 881282989b014e02af3db0131adc438e [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:05:43.956956 981 raft_consensus.cc:3037] T e6c828d6eac6488d9533a7d71561f7e0 P 82765766eda54d90b4657a8d9b80c88c [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:05:43.957692 1113 raft_consensus.cc:2454] T e6c828d6eac6488d9533a7d71561f7e0 P 881282989b014e02af3db0131adc438e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c64be363c6354f7d915f4a4bd93828af in term 1.
I20260923 15:05:43.957713 981 raft_consensus.cc:2454] T e6c828d6eac6488d9533a7d71561f7e0 P 82765766eda54d90b4657a8d9b80c88c [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate c64be363c6354f7d915f4a4bd93828af in term 1.
I20260923 15:05:43.957866 1177 leader_election.cc:302] T e6c828d6eac6488d9533a7d71561f7e0 P c64be363c6354f7d915f4a4bd93828af [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 881282989b014e02af3db0131adc438e, c64be363c6354f7d915f4a4bd93828af; no voters:
I20260923 15:05:43.957969 1310 raft_consensus.cc:2781] T e6c828d6eac6488d9533a7d71561f7e0 P c64be363c6354f7d915f4a4bd93828af [term 1 FOLLOWER]: Leader election won for term 1
I20260923 15:05:43.958132 1310 raft_consensus.cc:686] T e6c828d6eac6488d9533a7d71561f7e0 P c64be363c6354f7d915f4a4bd93828af [term 1 LEADER]: Becoming Leader. State: Replica: c64be363c6354f7d915f4a4bd93828af, State: Running, Role: LEADER
I20260923 15:05:43.958252 1310 consensus_queue.cc:237] T e6c828d6eac6488d9533a7d71561f7e0 P c64be363c6354f7d915f4a4bd93828af [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c64be363c6354f7d915f4a4bd93828af" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 41577 } } peers { permanent_uuid: "881282989b014e02af3db0131adc438e" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 44339 } } peers { permanent_uuid: "82765766eda54d90b4657a8d9b80c88c" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 35957 } }
I20260923 15:05:43.959195 838 catalog_manager.cc:5495] T e6c828d6eac6488d9533a7d71561f7e0 P c64be363c6354f7d915f4a4bd93828af reported cstate change: term changed from 0 to 1, leader changed from <none> to c64be363c6354f7d915f4a4bd93828af (127.13.75.131). New cstate: current_term: 1 leader_uuid: "c64be363c6354f7d915f4a4bd93828af" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "c64be363c6354f7d915f4a4bd93828af" member_type: VOTER last_known_addr { host: "127.13.75.131" port: 41577 } health_report { overall_health: HEALTHY } } peers { permanent_uuid: "881282989b014e02af3db0131adc438e" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 44339 } health_report { overall_health: UNKNOWN } } peers { permanent_uuid: "82765766eda54d90b4657a8d9b80c88c" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 35957 } health_report { overall_health: UNKNOWN } } }
I20260923 15:05:43.971184 1301 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:43.968622 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:53112 (local address 127.13.75.131:41577)
0923 15:05:43.968692 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0923 15:05:43.968694 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:05:43.968705 (+ 11us) server_negotiation.cc:407] Connection header received
0923 15:05:43.968745 (+ 40us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:05:43.968746 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:05:43.968765 (+ 19us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:05:43.968831 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 15:05:43.968984 (+ 153us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:05:43.969468 (+ 484us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:05:43.970299 (+ 831us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:05:43.970482 (+ 183us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:05:43.970552 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:05:43.970750 (+ 198us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 15:05:43.970866 (+ 116us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 15:05:43.970997 (+ 131us) server_negotiation.cc:1035] Waiting for connection context
0923 15:05:43.971014 (+ 17us) server_negotiation.cc:299] Negotiation successful
0923 15:05:43.971050 (+ 36us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":45}
I20260923 15:05:43.973136 981 raft_consensus.cc:1261] T e6c828d6eac6488d9533a7d71561f7e0 P 82765766eda54d90b4657a8d9b80c88c [term 1 FOLLOWER]: Refusing update from remote peer c64be363c6354f7d915f4a4bd93828af: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 15:05:43.973134 1113 raft_consensus.cc:1261] T e6c828d6eac6488d9533a7d71561f7e0 P 881282989b014e02af3db0131adc438e [term 1 FOLLOWER]: Refusing update from remote peer c64be363c6354f7d915f4a4bd93828af: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 15:05:43.973598 1314 consensus_queue.cc:1035] T e6c828d6eac6488d9533a7d71561f7e0 P c64be363c6354f7d915f4a4bd93828af [LEADER]: Connected to new peer: Peer: permanent_uuid: "82765766eda54d90b4657a8d9b80c88c" member_type: VOTER last_known_addr { host: "127.13.75.129" port: 35957 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 15:05:43.973747 1310 consensus_queue.cc:1035] T e6c828d6eac6488d9533a7d71561f7e0 P c64be363c6354f7d915f4a4bd93828af [LEADER]: Connected to new peer: Peer: permanent_uuid: "881282989b014e02af3db0131adc438e" member_type: VOTER last_known_addr { host: "127.13.75.130" port: 44339 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 15:05:43.977025 1318 mvcc.cc:205] Tried to move back new op lower bound from 7332560666511257600 to 7332560666453889024. Current Snapshot: MvccSnapshot[applied={T|T < 7332560666511257600 or (T in {7332560666511257600})}]
I20260923 15:05:43.977191 1319 mvcc.cc:205] Tried to move back new op lower bound from 7332560666511257600 to 7332560666453889024. Current Snapshot: MvccSnapshot[applied={T|T < 7332560666511257600}]
I20260923 15:05:43.978837 838 catalog_manager.cc:2461] Servicing SoftDeleteTable request from {username='slave'} at 127.0.0.1:45958:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 15:05:43.978960 838 catalog_manager.cc:2645] Servicing DeleteTable request from {username='slave'} at 127.0.0.1:45958:
table { table_name: "test-table" } modify_external_catalogs: true
I20260923 15:05:43.979773 838 catalog_manager.cc:5782] T 00000000000000000000000000000000 P e89ecef503384f98958d1067d5b7a1c9: Sending DeleteTablet for 3 replicas of tablet e6c828d6eac6488d9533a7d71561f7e0
I20260923 15:05:43.980132 1224 tablet_service.cc:1514] Processing DeleteTablet for tablet e6c828d6eac6488d9533a7d71561f7e0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 15:05:43 UTC) from {username='slave'} at 127.0.0.1:53104
I20260923 15:05:43.980157 961 tablet_service.cc:1514] Processing DeleteTablet for tablet e6c828d6eac6488d9533a7d71561f7e0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 15:05:43 UTC) from {username='slave'} at 127.0.0.1:49566
I20260923 15:05:43.980377 1326 tablet_replica.cc:330] T e6c828d6eac6488d9533a7d71561f7e0 P c64be363c6354f7d915f4a4bd93828af: stopping tablet replica
I20260923 15:05:43.980484 1326 raft_consensus.cc:2229] T e6c828d6eac6488d9533a7d71561f7e0 P c64be363c6354f7d915f4a4bd93828af [term 1 LEADER]: Raft consensus shutting down.
I20260923 15:05:43.980508 1092 tablet_service.cc:1514] Processing DeleteTablet for tablet e6c828d6eac6488d9533a7d71561f7e0 with delete_type TABLET_DATA_DELETED (Table deleted at 2026-09-23 15:05:43 UTC) from {username='slave'} at 127.0.0.1:40296
I20260923 15:05:43.980772 1327 tablet_replica.cc:330] T e6c828d6eac6488d9533a7d71561f7e0 P 82765766eda54d90b4657a8d9b80c88c: stopping tablet replica
I20260923 15:05:43.980829 13614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKC4soy/build/release/bin/kudu with pid 899
I20260923 15:05:43.980882 1327 raft_consensus.cc:2229] T e6c828d6eac6488d9533a7d71561f7e0 P 82765766eda54d90b4657a8d9b80c88c [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 15:05:43.980997 1327 raft_consensus.cc:2258] T e6c828d6eac6488d9533a7d71561f7e0 P 82765766eda54d90b4657a8d9b80c88c [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 15:05:43.981159 1328 tablet_replica.cc:330] T e6c828d6eac6488d9533a7d71561f7e0 P 881282989b014e02af3db0131adc438e: stopping tablet replica
I20260923 15:05:43.981264 1328 raft_consensus.cc:2229] T e6c828d6eac6488d9533a7d71561f7e0 P 881282989b014e02af3db0131adc438e [term 1 FOLLOWER]: Raft consensus shutting down.
I20260923 15:05:43.981361 1328 raft_consensus.cc:2258] T e6c828d6eac6488d9533a7d71561f7e0 P 881282989b014e02af3db0131adc438e [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 15:05:43.982153 1328 ts_tablet_manager.cc:1905] T e6c828d6eac6488d9533a7d71561f7e0 P 881282989b014e02af3db0131adc438e: Deleting tablet data with delete state TABLET_DATA_DELETED
I20260923 15:05:43.983980 1328 ts_tablet_manager.cc:1918] T e6c828d6eac6488d9533a7d71561f7e0 P 881282989b014e02af3db0131adc438e: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 15:05:43.984064 1328 log.cc:1196] T e6c828d6eac6488d9533a7d71561f7e0 P 881282989b014e02af3db0131adc438e: Deleting WAL directory at /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790175691491353-13614-0/minicluster-data/ts-1/wal/wals/e6c828d6eac6488d9533a7d71561f7e0
I20260923 15:05:43.984326 1328 ts_tablet_manager.cc:1939] T e6c828d6eac6488d9533a7d71561f7e0 P 881282989b014e02af3db0131adc438e: Deleting consensus metadata
I20260923 15:05:43.984768 823 catalog_manager.cc:4841] TS 881282989b014e02af3db0131adc438e (127.13.75.130:44339): tablet e6c828d6eac6488d9533a7d71561f7e0 (table test-table [id=910e201d37c24026b18dafe8af9fadf6]) successfully deleted
I20260923 15:05:43.980588 1326 raft_consensus.cc:2258] T e6c828d6eac6488d9533a7d71561f7e0 P c64be363c6354f7d915f4a4bd93828af [term 1 FOLLOWER]: Raft consensus is shut down!
I20260923 15:05:43.987498 1326 ts_tablet_manager.cc:1905] T e6c828d6eac6488d9533a7d71561f7e0 P c64be363c6354f7d915f4a4bd93828af: Deleting tablet data with delete state TABLET_DATA_DELETED
W20260923 15:05:43.988351 824 proxy.cc:239] Call had error, refreshing address and retrying: Network error: recv got EOF from 127.13.75.129:35957 (error 108)
I20260923 15:05:43.988586 13614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKC4soy/build/release/bin/kudu with pid 1030
I20260923 15:05:43.989564 1300 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:43.988746 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.129:35957 (local address 127.0.0.1:49582)
0923 15:05:43.989489 (+ 743us) negotiation.cc:107] Waiting for socket to connect
0923 15:05:43.989532 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.129:35957: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":720}
W20260923 15:05:43.989719 824 catalog_manager.cc:4584] TS 82765766eda54d90b4657a8d9b80c88c (127.13.75.129:35957): DeleteTablet:TABLET_DATA_DELETED RPC failed for tablet e6c828d6eac6488d9533a7d71561f7e0: Network error: Client connection negotiation failed: client connection to 127.13.75.129:35957: connect: Connection refused (error 111)
I20260923 15:05:43.993117 1326 ts_tablet_manager.cc:1918] T e6c828d6eac6488d9533a7d71561f7e0 P c64be363c6354f7d915f4a4bd93828af: tablet deleted with delete type TABLET_DATA_DELETED: last-logged OpId 1.2
I20260923 15:05:43.993223 1326 log.cc:1196] T e6c828d6eac6488d9533a7d71561f7e0 P c64be363c6354f7d915f4a4bd93828af: Deleting WAL directory at /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.EncryptionPolicyTest.TestEncryptionPolicy_5.1790175691491353-13614-0/minicluster-data/ts-2/wal/wals/e6c828d6eac6488d9533a7d71561f7e0
I20260923 15:05:43.993513 1326 ts_tablet_manager.cc:1939] T e6c828d6eac6488d9533a7d71561f7e0 P c64be363c6354f7d915f4a4bd93828af: Deleting consensus metadata
I20260923 15:05:43.994020 825 catalog_manager.cc:4841] TS c64be363c6354f7d915f4a4bd93828af (127.13.75.131:41577): tablet e6c828d6eac6488d9533a7d71561f7e0 (table test-table [id=910e201d37c24026b18dafe8af9fadf6]) successfully deleted
I20260923 15:05:43.995210 13614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKC4soy/build/release/bin/kudu with pid 1162
I20260923 15:05:44.001240 13614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKC4soy/build/release/bin/kudu with pid 808
2026-09-23T15:05:44Z chronyd exiting
[ OK ] EncryptionPolicyTest.TestEncryptionPolicy/5 (755 ms)
[----------] 6 tests from EncryptionPolicyTest (9029 ms total)
[----------] 18 tests from AuthTokenIssuingTest
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/0
Loading random data
Initializing database '/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790175691491353-13614-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 15:05:44 dist-test-slave-cnb0 krb5kdc[1332](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 15:05:44 dist-test-slave-cnb0 krb5kdc[1332](info): set up 2 sockets
Sep 23 15:05:44 dist-test-slave-cnb0 krb5kdc[1332](info): commencing operation
krb5kdc: starting...
W20260923 15:05:45.056129 13614 mini_kdc.cc:121] Time spent starting KDC: real 1.034s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790175691491353-13614-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790175691491353-13614-0/krb5kdc/test-user.keytab.
Sep 23 15:05:45 dist-test-slave-cnb0 krb5kdc[1332](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175945, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T15:05:45Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T15:05:45Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790175691491353-13614-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790175691491353-13614-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260923 15:05:45.183050 13614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKC4soy/build/release/bin/kudu
/tmp/dist-test-taskKC4soy/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790175691491353-13614-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790175691491353-13614-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790175691491353-13614-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790175691491353-13614-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34613
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790175691491353-13614-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39159
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34613
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790175691491353-13614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790175691491353-13614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790175691491353-13614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:05:45.266942 1345 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:05:45.267215 1345 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:05:45.267257 1345 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:05:45.268507 1345 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 15:05:45.268590 1345 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:05:45.268618 1345 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:05:45.268652 1345 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 15:05:45.268679 1345 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 15:05:45.270309 1345 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39159
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790175691491353-13614-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790175691491353-13614-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34613
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790175691491353-13614-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34613
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790175691491353-13614-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1345
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790175691491353-13614-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:05:45.270619 1345 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:05:45.270937 1345 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:05:45.276027 1351 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:05:45.276129 1350 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:05:45.276027 1353 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:05:45.276437 1345 server_base.cc:1034] running on GCE node
I20260923 15:05:45.276700 1345 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:05:45.277031 1345 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:05:45.278208 1345 hybrid_clock.cc:648] HybridClock initialized: now 1790175945278192 us; error 35 us; skew 500 ppm
Sep 23 15:05:45 dist-test-slave-cnb0 krb5kdc[1332](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175945, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:05:45.285584 1345 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260923 15:05:45.286073 1345 webserver.cc:460] Webserver started at http://127.0.0.1:34171/ using document root <none> and password file <none>
I20260923 15:05:45.286352 1345 fs_manager.cc:359] Metadata directory not provided
I20260923 15:05:45.286406 1345 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:05:45.286513 1345 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:05:45.287842 1345 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790175691491353-13614-0/minicluster-data/master-0/data/instance:
uuid: "bd51b43e694645ec9683f349aa4addb8"
format_stamp: "Formatted at 2026-09-23 15:05:45 on dist-test-slave-cnb0"
server_key: "e9a6ba523ae7fb753d446948f93d8924"
server_key_iv: "1820fb6375ee62f6c104396ac2e53477"
server_key_version: "encryptionkey@0"
I20260923 15:05:45.288189 1345 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790175691491353-13614-0/minicluster-data/master-0/wal/instance:
uuid: "bd51b43e694645ec9683f349aa4addb8"
format_stamp: "Formatted at 2026-09-23 15:05:45 on dist-test-slave-cnb0"
server_key: "e9a6ba523ae7fb753d446948f93d8924"
server_key_iv: "1820fb6375ee62f6c104396ac2e53477"
server_key_version: "encryptionkey@0"
I20260923 15:05:45.289695 1345 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260923 15:05:45.290530 1360 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:05:45.290735 1345 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260923 15:05:45.290841 1345 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790175691491353-13614-0/minicluster-data/master-0/data,/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790175691491353-13614-0/minicluster-data/master-0/wal
uuid: "bd51b43e694645ec9683f349aa4addb8"
format_stamp: "Formatted at 2026-09-23 15:05:45 on dist-test-slave-cnb0"
server_key: "e9a6ba523ae7fb753d446948f93d8924"
server_key_iv: "1820fb6375ee62f6c104396ac2e53477"
server_key_version: "encryptionkey@0"
I20260923 15:05:45.290967 1345 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790175691491353-13614-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790175691491353-13614-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790175691491353-13614-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:05:45.301326 1345 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:05:45.303750 1345 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:05:45.303867 1345 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:05:45.307816 1345 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34613
I20260923 15:05:45.307852 1412 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34613 every 8 connection(s)
I20260923 15:05:45.308291 1345 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790175691491353-13614-0/minicluster-data/master-0/data/info.pb
I20260923 15:05:45.309291 1413 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:05:45.311892 1413 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bd51b43e694645ec9683f349aa4addb8: Bootstrap starting.
I20260923 15:05:45.312579 1413 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P bd51b43e694645ec9683f349aa4addb8: Neither blocks nor log segments found. Creating new log.
I20260923 15:05:45.312899 1413 log.cc:824] T 00000000000000000000000000000000 P bd51b43e694645ec9683f349aa4addb8: Log is configured to *not* fsync() on all Append() calls
I20260923 15:05:45.313665 1413 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P bd51b43e694645ec9683f349aa4addb8: No bootstrap required, opened a new log
I20260923 15:05:45.315632 1413 raft_consensus.cc:348] T 00000000000000000000000000000000 P bd51b43e694645ec9683f349aa4addb8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd51b43e694645ec9683f349aa4addb8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34613 } }
I20260923 15:05:45.315783 1413 raft_consensus.cc:374] T 00000000000000000000000000000000 P bd51b43e694645ec9683f349aa4addb8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:05:45.315809 1413 raft_consensus.cc:729] T 00000000000000000000000000000000 P bd51b43e694645ec9683f349aa4addb8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: bd51b43e694645ec9683f349aa4addb8, State: Initialized, Role: FOLLOWER
I20260923 15:05:45.315981 1413 consensus_queue.cc:260] T 00000000000000000000000000000000 P bd51b43e694645ec9683f349aa4addb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd51b43e694645ec9683f349aa4addb8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34613 } }
I20260923 15:05:45.316061 1413 raft_consensus.cc:388] T 00000000000000000000000000000000 P bd51b43e694645ec9683f349aa4addb8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 15:05:45.316084 1413 raft_consensus.cc:482] T 00000000000000000000000000000000 P bd51b43e694645ec9683f349aa4addb8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 15:05:45.316160 1413 raft_consensus.cc:3037] T 00000000000000000000000000000000 P bd51b43e694645ec9683f349aa4addb8 [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:05:45.316767 1413 raft_consensus.cc:504] T 00000000000000000000000000000000 P bd51b43e694645ec9683f349aa4addb8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd51b43e694645ec9683f349aa4addb8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34613 } }
I20260923 15:05:45.316885 1413 leader_election.cc:302] T 00000000000000000000000000000000 P bd51b43e694645ec9683f349aa4addb8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: bd51b43e694645ec9683f349aa4addb8; no voters:
I20260923 15:05:45.317095 1413 leader_election.cc:288] T 00000000000000000000000000000000 P bd51b43e694645ec9683f349aa4addb8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 15:05:45.317131 1416 raft_consensus.cc:2781] T 00000000000000000000000000000000 P bd51b43e694645ec9683f349aa4addb8 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 15:05:45.317306 1416 raft_consensus.cc:686] T 00000000000000000000000000000000 P bd51b43e694645ec9683f349aa4addb8 [term 1 LEADER]: Becoming Leader. State: Replica: bd51b43e694645ec9683f349aa4addb8, State: Running, Role: LEADER
I20260923 15:05:45.317385 1413 sys_catalog.cc:564] T 00000000000000000000000000000000 P bd51b43e694645ec9683f349aa4addb8 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 15:05:45.317400 1416 consensus_queue.cc:237] T 00000000000000000000000000000000 P bd51b43e694645ec9683f349aa4addb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd51b43e694645ec9683f349aa4addb8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34613 } }
I20260923 15:05:45.317873 1417 sys_catalog.cc:455] T 00000000000000000000000000000000 P bd51b43e694645ec9683f349aa4addb8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "bd51b43e694645ec9683f349aa4addb8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd51b43e694645ec9683f349aa4addb8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34613 } } }
I20260923 15:05:45.317960 1417 sys_catalog.cc:458] T 00000000000000000000000000000000 P bd51b43e694645ec9683f349aa4addb8 [sys.catalog]: This master's current role is: LEADER
I20260923 15:05:45.318027 1418 sys_catalog.cc:455] T 00000000000000000000000000000000 P bd51b43e694645ec9683f349aa4addb8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader bd51b43e694645ec9683f349aa4addb8. Latest consensus state: current_term: 1 leader_uuid: "bd51b43e694645ec9683f349aa4addb8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "bd51b43e694645ec9683f349aa4addb8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34613 } } }
I20260923 15:05:45.318346 13614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKC4soy/build/release/bin/kudu as pid 1345
I20260923 15:05:45.318472 13614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_0.1790175691491353-13614-0/minicluster-data/master-0/wal/instance
I20260923 15:05:45.318624 1423 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 15:05:45.318739 13614 external_mini_cluster.cc:1428] Setting key c38c907810cdd15f176e4362d317a30e
I20260923 15:05:45.318944 1418 sys_catalog.cc:458] T 00000000000000000000000000000000 P bd51b43e694645ec9683f349aa4addb8 [sys.catalog]: This master's current role is: LEADER
I20260923 15:05:45.319199 1423 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 15:05:45.320886 1423 catalog_manager.cc:1347] Generated new cluster ID: ad85574ac8804625ab0ff15a7ca9d4b0
I20260923 15:05:45.320936 1423 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 15:05:45.326216 1423 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 15:05:45.326984 1423 catalog_manager.cc:1504] Loading token signing keys...
Sep 23 15:05:45 dist-test-slave-cnb0 krb5kdc[1332](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175945, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260923 15:05:45.335901 1423 catalog_manager.cc:5868] T 00000000000000000000000000000000 P bd51b43e694645ec9683f349aa4addb8: Generated new TSK 0
I20260923 15:05:45.336077 1423 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 15:05:45.336349 1434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:45.319327 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52561 (local address 127.0.0.1:34613)
0923 15:05:45.319852 (+ 525us) server_negotiation.cc:206] Beginning negotiation
0923 15:05:45.319863 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0923 15:05:45.319895 (+ 32us) server_negotiation.cc:407] Connection header received
0923 15:05:45.320025 (+ 130us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:05:45.320033 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:05:45.320094 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:05:45.320242 (+ 148us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:05:45.325424 (+ 5182us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:05:45.327149 (+ 1725us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:05:45.328230 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:05:45.328912 (+ 682us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:05:45.329039 (+ 127us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:05:45.332135 (+ 3096us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:05:45.332140 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:05:45.332141 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:05:45.332155 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:05:45.334198 (+ 2043us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:05:45.334730 (+ 532us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:05:45.334733 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:05:45.334734 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:05:45.334853 (+ 119us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:05:45.335262 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:05:45.335263 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:05:45.335263 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:05:45.335534 (+ 271us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:05:45.335722 (+ 188us) server_negotiation.cc:1035] Waiting for connection context
0923 15:05:45.335895 (+ 173us) server_negotiation.cc:299] Negotiation successful
0923 15:05:45.336098 (+ 203us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":471,"thread_start_us":96,"threads_started":1}
I20260923 15:05:45.337374 13614 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 15:05:45.344980 1434 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:45.338720 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56085 (local address 127.0.0.1:34613)
0923 15:05:45.338761 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0923 15:05:45.338763 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:05:45.338919 (+ 156us) server_negotiation.cc:407] Connection header received
0923 15:05:45.338992 (+ 73us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:05:45.338993 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:05:45.339002 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:05:45.339058 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:05:45.339584 (+ 526us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:05:45.340125 (+ 541us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:05:45.340858 (+ 733us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:05:45.341095 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:05:45.341184 (+ 89us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:05:45.342462 (+ 1278us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:05:45.342464 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:05:45.342464 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:05:45.342472 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:05:45.343799 (+ 1327us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:05:45.344163 (+ 364us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:05:45.344164 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:05:45.344165 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:05:45.344221 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:05:45.344449 (+ 228us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:05:45.344450 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:05:45.344451 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:05:45.344561 (+ 110us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:05:45.344684 (+ 123us) server_negotiation.cc:1035] Waiting for connection context
0923 15:05:45.344804 (+ 120us) server_negotiation.cc:299] Negotiation successful
0923 15:05:45.344868 (+ 64us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260923 15:05:45.347311 13614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKC4soy/build/release/bin/kudu with pid 1345
W20260923 15:05:45.351876 1330 connection.cc:535] client connection to 127.0.0.1:34613 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:34613): Connection reset by peer (error 104)
2026-09-23T15:05:45Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/0 (1343 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/1
Loading random data
Initializing database '/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790175691491353-13614-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 15:05:45 dist-test-slave-cnb0 krb5kdc[1444](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 15:05:45 dist-test-slave-cnb0 krb5kdc[1444](info): set up 2 sockets
Sep 23 15:05:45 dist-test-slave-cnb0 krb5kdc[1444](info): commencing operation
krb5kdc: starting...
W20260923 15:05:46.396310 13614 mini_kdc.cc:121] Time spent starting KDC: real 1.030s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790175691491353-13614-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790175691491353-13614-0/krb5kdc/test-user.keytab.
Sep 23 15:05:46 dist-test-slave-cnb0 krb5kdc[1444](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175946, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T15:05:46Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T15:05:46Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790175691491353-13614-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790175691491353-13614-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260923 15:05:46.522478 13614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKC4soy/build/release/bin/kudu
/tmp/dist-test-taskKC4soy/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790175691491353-13614-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790175691491353-13614-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790175691491353-13614-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790175691491353-13614-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38297
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790175691491353-13614-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38343
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:38297
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790175691491353-13614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790175691491353-13614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790175691491353-13614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:05:46.606282 1457 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:05:46.606529 1457 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:05:46.606555 1457 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:05:46.607869 1457 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 15:05:46.607919 1457 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:05:46.607934 1457 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:05:46.607947 1457 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 15:05:46.607959 1457 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 15:05:46.609539 1457 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38343
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790175691491353-13614-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790175691491353-13614-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:38297
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790175691491353-13614-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:38297
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790175691491353-13614-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1457
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790175691491353-13614-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:05:46.609813 1457 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:05:46.610087 1457 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:05:46.616151 1462 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:05:46.616237 1465 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:05:46.616187 1463 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:05:46.616462 1457 server_base.cc:1034] running on GCE node
I20260923 15:05:46.616734 1457 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:05:46.617126 1457 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:05:46.618305 1457 hybrid_clock.cc:648] HybridClock initialized: now 1790175946618289 us; error 28 us; skew 500 ppm
Sep 23 15:05:46 dist-test-slave-cnb0 krb5kdc[1444](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175946, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:05:46.621009 1457 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260923 15:05:46.621517 1457 webserver.cc:460] Webserver started at http://127.0.0.1:38677/ using document root <none> and password file <none>
I20260923 15:05:46.621791 1457 fs_manager.cc:359] Metadata directory not provided
I20260923 15:05:46.621840 1457 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:05:46.622016 1457 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:05:46.623211 1457 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790175691491353-13614-0/minicluster-data/master-0/data/instance:
uuid: "65e89e75726940579e32216c15c26019"
format_stamp: "Formatted at 2026-09-23 15:05:46 on dist-test-slave-cnb0"
server_key: "38d77a73d0a6ffbabe5c8873c70defa6"
server_key_iv: "39ad3a4afa4c5a755ae1215aa3020f09"
server_key_version: "encryptionkey@0"
I20260923 15:05:46.623615 1457 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790175691491353-13614-0/minicluster-data/master-0/wal/instance:
uuid: "65e89e75726940579e32216c15c26019"
format_stamp: "Formatted at 2026-09-23 15:05:46 on dist-test-slave-cnb0"
server_key: "38d77a73d0a6ffbabe5c8873c70defa6"
server_key_iv: "39ad3a4afa4c5a755ae1215aa3020f09"
server_key_version: "encryptionkey@0"
I20260923 15:05:46.625059 1457 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.001s
I20260923 15:05:46.626101 1472 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:05:46.626289 1457 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:05:46.626403 1457 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790175691491353-13614-0/minicluster-data/master-0/data,/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790175691491353-13614-0/minicluster-data/master-0/wal
uuid: "65e89e75726940579e32216c15c26019"
format_stamp: "Formatted at 2026-09-23 15:05:46 on dist-test-slave-cnb0"
server_key: "38d77a73d0a6ffbabe5c8873c70defa6"
server_key_iv: "39ad3a4afa4c5a755ae1215aa3020f09"
server_key_version: "encryptionkey@0"
I20260923 15:05:46.626487 1457 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790175691491353-13614-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790175691491353-13614-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790175691491353-13614-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:05:46.644093 1457 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:05:46.646744 1457 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:05:46.647020 1457 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:05:46.653761 1457 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:38297
I20260923 15:05:46.653802 1524 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:38297 every 8 connection(s)
I20260923 15:05:46.654302 1457 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790175691491353-13614-0/minicluster-data/master-0/data/info.pb
I20260923 15:05:46.655853 1525 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:05:46.657850 13614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKC4soy/build/release/bin/kudu as pid 1457
I20260923 15:05:46.657966 13614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_1.1790175691491353-13614-0/minicluster-data/master-0/wal/instance
I20260923 15:05:46.658165 13614 external_mini_cluster.cc:1428] Setting key 12fd5059fa8cd5909476a259ed27c58c
I20260923 15:05:46.659492 1525 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 65e89e75726940579e32216c15c26019: Bootstrap starting.
I20260923 15:05:46.660159 1525 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 65e89e75726940579e32216c15c26019: Neither blocks nor log segments found. Creating new log.
I20260923 15:05:46.660516 1525 log.cc:824] T 00000000000000000000000000000000 P 65e89e75726940579e32216c15c26019: Log is configured to *not* fsync() on all Append() calls
I20260923 15:05:46.661613 1525 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 65e89e75726940579e32216c15c26019: No bootstrap required, opened a new log
I20260923 15:05:46.664743 1525 raft_consensus.cc:348] T 00000000000000000000000000000000 P 65e89e75726940579e32216c15c26019 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65e89e75726940579e32216c15c26019" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38297 } }
I20260923 15:05:46.664937 1525 raft_consensus.cc:374] T 00000000000000000000000000000000 P 65e89e75726940579e32216c15c26019 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:05:46.664989 1525 raft_consensus.cc:729] T 00000000000000000000000000000000 P 65e89e75726940579e32216c15c26019 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 65e89e75726940579e32216c15c26019, State: Initialized, Role: FOLLOWER
I20260923 15:05:46.665138 1525 consensus_queue.cc:260] T 00000000000000000000000000000000 P 65e89e75726940579e32216c15c26019 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65e89e75726940579e32216c15c26019" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38297 } }
I20260923 15:05:46.665241 1525 raft_consensus.cc:388] T 00000000000000000000000000000000 P 65e89e75726940579e32216c15c26019 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 15:05:46.665285 1525 raft_consensus.cc:482] T 00000000000000000000000000000000 P 65e89e75726940579e32216c15c26019 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 15:05:46.665333 1525 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 65e89e75726940579e32216c15c26019 [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:05:46.666033 1528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:46.659263 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41345 (local address 127.0.0.1:38297)
0923 15:05:46.659465 (+ 202us) server_negotiation.cc:206] Beginning negotiation
0923 15:05:46.659474 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0923 15:05:46.659503 (+ 29us) server_negotiation.cc:407] Connection header received
0923 15:05:46.659985 (+ 482us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:05:46.659993 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:05:46.660044 (+ 51us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:05:46.660308 (+ 264us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:05:46.660492 (+ 184us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:05:46.662188 (+ 1696us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:05:46.663442 (+ 1254us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:05:46.664152 (+ 710us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:05:46.664285 (+ 133us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:05:46.665267 (+ 982us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:05:46.665272 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:05:46.665273 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 15:05:46.665498 (+ 225us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:05:46.665541 (+ 43us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 15:05:46.665558 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:05:46.665654 (+ 96us) server_negotiation.cc:1035] Waiting for connection context
0923 15:05:46.665774 (+ 120us) server_negotiation.cc:299] Negotiation successful
0923 15:05:46.665804 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":135,"thread_start_us":92,"threads_started":1}
I20260923 15:05:46.666229 1525 raft_consensus.cc:504] T 00000000000000000000000000000000 P 65e89e75726940579e32216c15c26019 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65e89e75726940579e32216c15c26019" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38297 } }
I20260923 15:05:46.666342 1525 leader_election.cc:302] T 00000000000000000000000000000000 P 65e89e75726940579e32216c15c26019 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 65e89e75726940579e32216c15c26019; no voters:
I20260923 15:05:46.666520 1525 leader_election.cc:288] T 00000000000000000000000000000000 P 65e89e75726940579e32216c15c26019 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 15:05:46.666564 1530 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 65e89e75726940579e32216c15c26019 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 15:05:46.666687 1530 raft_consensus.cc:686] T 00000000000000000000000000000000 P 65e89e75726940579e32216c15c26019 [term 1 LEADER]: Becoming Leader. State: Replica: 65e89e75726940579e32216c15c26019, State: Running, Role: LEADER
I20260923 15:05:46.666770 1530 consensus_queue.cc:237] T 00000000000000000000000000000000 P 65e89e75726940579e32216c15c26019 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65e89e75726940579e32216c15c26019" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38297 } }
I20260923 15:05:46.667214 1531 sys_catalog.cc:455] T 00000000000000000000000000000000 P 65e89e75726940579e32216c15c26019 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "65e89e75726940579e32216c15c26019" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65e89e75726940579e32216c15c26019" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38297 } } }
I20260923 15:05:46.667332 1531 sys_catalog.cc:458] T 00000000000000000000000000000000 P 65e89e75726940579e32216c15c26019 [sys.catalog]: This master's current role is: LEADER
I20260923 15:05:46.666800 1525 sys_catalog.cc:564] T 00000000000000000000000000000000 P 65e89e75726940579e32216c15c26019 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 15:05:46.667502 1532 sys_catalog.cc:455] T 00000000000000000000000000000000 P 65e89e75726940579e32216c15c26019 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 65e89e75726940579e32216c15c26019. Latest consensus state: current_term: 1 leader_uuid: "65e89e75726940579e32216c15c26019" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "65e89e75726940579e32216c15c26019" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 38297 } } }
I20260923 15:05:46.667570 1532 sys_catalog.cc:458] T 00000000000000000000000000000000 P 65e89e75726940579e32216c15c26019 [sys.catalog]: This master's current role is: LEADER
I20260923 15:05:46.667686 1534 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 15:05:46.667809 13614 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 15:05:46.668283 1534 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 15:05:46.670053 1534 catalog_manager.cc:1347] Generated new cluster ID: 095f5d77015048879f312a875d4a52d1
I20260923 15:05:46.670114 1534 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 15:05:46.674122 1528 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:46.669941 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40433 (local address 127.0.0.1:38297)
0923 15:05:46.669977 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0923 15:05:46.669979 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:05:46.670226 (+ 247us) server_negotiation.cc:407] Connection header received
0923 15:05:46.670293 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:05:46.670294 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:05:46.670305 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:05:46.670370 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:05:46.670571 (+ 201us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:05:46.671355 (+ 784us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:05:46.672673 (+ 1318us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:05:46.672979 (+ 306us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:05:46.673015 (+ 36us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:05:46.673834 (+ 819us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:05:46.673836 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:05:46.673836 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 15:05:46.673842 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:05:46.673861 (+ 19us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 15:05:46.673865 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:05:46.673938 (+ 73us) server_negotiation.cc:1035] Waiting for connection context
0923 15:05:46.674013 (+ 75us) server_negotiation.cc:299] Negotiation successful
0923 15:05:46.674040 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260923 15:05:46.699715 1534 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 15:05:46.700515 1534 catalog_manager.cc:1504] Loading token signing keys...
I20260923 15:05:46.704576 1534 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 65e89e75726940579e32216c15c26019: Generated new TSK 0
I20260923 15:05:46.704779 1534 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 15:05:46.708978 13614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKC4soy/build/release/bin/kudu with pid 1457
W20260923 15:05:46.714057 1442 connection.cc:535] client connection to 127.0.0.1:38297 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:38297): Connection reset by peer (error 104)
2026-09-23T15:05:46Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/1 (1361 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/2
Loading random data
Initializing database '/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790175691491353-13614-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 15:05:46 dist-test-slave-cnb0 krb5kdc[1556](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 15:05:46 dist-test-slave-cnb0 krb5kdc[1556](info): set up 2 sockets
Sep 23 15:05:46 dist-test-slave-cnb0 krb5kdc[1556](info): commencing operation
krb5kdc: starting...
W20260923 15:05:47.768829 13614 mini_kdc.cc:121] Time spent starting KDC: real 1.041s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790175691491353-13614-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790175691491353-13614-0/krb5kdc/test-user.keytab.
Sep 23 15:05:47 dist-test-slave-cnb0 krb5kdc[1556](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175947, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T15:05:47Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T15:05:47Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790175691491353-13614-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790175691491353-13614-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260923 15:05:47.898376 13614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKC4soy/build/release/bin/kudu
/tmp/dist-test-taskKC4soy/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790175691491353-13614-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790175691491353-13614-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790175691491353-13614-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790175691491353-13614-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37147
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790175691491353-13614-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:38375
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:37147
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790175691491353-13614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790175691491353-13614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790175691491353-13614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:05:47.980726 1569 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:05:47.980950 1569 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:05:47.980975 1569 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:05:47.982206 1569 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 15:05:47.982250 1569 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:05:47.982266 1569 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:05:47.982280 1569 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 15:05:47.982291 1569 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 15:05:47.983898 1569 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:38375
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790175691491353-13614-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790175691491353-13614-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:37147
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790175691491353-13614-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:37147
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790175691491353-13614-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1569
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790175691491353-13614-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:05:47.984107 1569 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:05:47.984289 1569 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:05:47.989480 1575 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:05:47.989589 1569 server_base.cc:1034] running on GCE node
W20260923 15:05:47.989646 1574 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:05:47.989646 1577 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:05:47.990049 1569 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:05:47.990381 1569 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:05:47.991561 1569 hybrid_clock.cc:648] HybridClock initialized: now 1790175947991545 us; error 49 us; skew 500 ppm
Sep 23 15:05:47 dist-test-slave-cnb0 krb5kdc[1556](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175947, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:05:47.994256 1569 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260923 15:05:47.994823 1569 webserver.cc:460] Webserver started at http://127.0.0.1:37941/ using document root <none> and password file <none>
I20260923 15:05:47.995126 1569 fs_manager.cc:359] Metadata directory not provided
I20260923 15:05:47.995208 1569 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:05:47.995363 1569 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:05:47.996619 1569 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790175691491353-13614-0/minicluster-data/master-0/data/instance:
uuid: "b2e62979304d48759f47641d4ce01bb8"
format_stamp: "Formatted at 2026-09-23 15:05:47 on dist-test-slave-cnb0"
server_key: "1ec98e1bb0a0931579477079598cc109"
server_key_iv: "a3660249963ab07ccaa871369b087837"
server_key_version: "encryptionkey@0"
I20260923 15:05:47.997047 1569 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790175691491353-13614-0/minicluster-data/master-0/wal/instance:
uuid: "b2e62979304d48759f47641d4ce01bb8"
format_stamp: "Formatted at 2026-09-23 15:05:47 on dist-test-slave-cnb0"
server_key: "1ec98e1bb0a0931579477079598cc109"
server_key_iv: "a3660249963ab07ccaa871369b087837"
server_key_version: "encryptionkey@0"
I20260923 15:05:47.998482 1569 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 15:05:47.999485 1584 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:05:47.999770 1569 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260923 15:05:47.999902 1569 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790175691491353-13614-0/minicluster-data/master-0/data,/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790175691491353-13614-0/minicluster-data/master-0/wal
uuid: "b2e62979304d48759f47641d4ce01bb8"
format_stamp: "Formatted at 2026-09-23 15:05:47 on dist-test-slave-cnb0"
server_key: "1ec98e1bb0a0931579477079598cc109"
server_key_iv: "a3660249963ab07ccaa871369b087837"
server_key_version: "encryptionkey@0"
I20260923 15:05:47.999998 1569 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790175691491353-13614-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790175691491353-13614-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790175691491353-13614-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:05:48.003707 1569 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:05:48.005921 1569 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:05:48.006186 1569 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:05:48.010542 1569 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:37147
I20260923 15:05:48.010576 1636 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:37147 every 8 connection(s)
I20260923 15:05:48.011052 1569 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790175691491353-13614-0/minicluster-data/master-0/data/info.pb
I20260923 15:05:48.011967 1637 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:05:48.013384 13614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKC4soy/build/release/bin/kudu as pid 1569
I20260923 15:05:48.013513 13614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_2.1790175691491353-13614-0/minicluster-data/master-0/wal/instance
I20260923 15:05:48.013728 13614 external_mini_cluster.cc:1428] Setting key 34e3a4319a8ab93f536d5a5373a6eb23
I20260923 15:05:48.014659 1637 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b2e62979304d48759f47641d4ce01bb8: Bootstrap starting.
I20260923 15:05:48.015395 1637 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P b2e62979304d48759f47641d4ce01bb8: Neither blocks nor log segments found. Creating new log.
I20260923 15:05:48.016060 1637 log.cc:824] T 00000000000000000000000000000000 P b2e62979304d48759f47641d4ce01bb8: Log is configured to *not* fsync() on all Append() calls
I20260923 15:05:48.016474 1640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:48.014294 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:60225 (local address 127.0.0.1:37147)
0923 15:05:48.014504 (+ 210us) server_negotiation.cc:206] Beginning negotiation
0923 15:05:48.014514 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0923 15:05:48.014546 (+ 32us) server_negotiation.cc:407] Connection header received
0923 15:05:48.015138 (+ 592us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:05:48.015147 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:05:48.015180 (+ 33us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:05:48.015348 (+ 168us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:05:48.015986 (+ 638us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:05:48.015990 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:05:48.015990 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 15:05:48.015995 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:05:48.016039 (+ 44us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 15:05:48.016053 (+ 14us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:05:48.016146 (+ 93us) server_negotiation.cc:1035] Waiting for connection context
0923 15:05:48.016219 (+ 73us) server_negotiation.cc:299] Negotiation successful
0923 15:05:48.016264 (+ 45us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":150,"thread_start_us":93,"threads_started":1}
I20260923 15:05:48.016875 1637 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P b2e62979304d48759f47641d4ce01bb8: No bootstrap required, opened a new log
I20260923 15:05:48.017505 13614 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 15:05:48.019127 1637 raft_consensus.cc:348] T 00000000000000000000000000000000 P b2e62979304d48759f47641d4ce01bb8 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2e62979304d48759f47641d4ce01bb8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37147 } }
I20260923 15:05:48.019320 1637 raft_consensus.cc:374] T 00000000000000000000000000000000 P b2e62979304d48759f47641d4ce01bb8 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:05:48.019368 1637 raft_consensus.cc:729] T 00000000000000000000000000000000 P b2e62979304d48759f47641d4ce01bb8 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: b2e62979304d48759f47641d4ce01bb8, State: Initialized, Role: FOLLOWER
I20260923 15:05:48.019495 1637 consensus_queue.cc:260] T 00000000000000000000000000000000 P b2e62979304d48759f47641d4ce01bb8 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2e62979304d48759f47641d4ce01bb8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37147 } }
I20260923 15:05:48.019619 1637 raft_consensus.cc:388] T 00000000000000000000000000000000 P b2e62979304d48759f47641d4ce01bb8 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 15:05:48.019656 1637 raft_consensus.cc:482] T 00000000000000000000000000000000 P b2e62979304d48759f47641d4ce01bb8 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 15:05:48.019712 1637 raft_consensus.cc:3037] T 00000000000000000000000000000000 P b2e62979304d48759f47641d4ce01bb8 [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:05:48.019989 1640 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:48.018863 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41731 (local address 127.0.0.1:37147)
0923 15:05:48.018922 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0923 15:05:48.018924 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:05:48.018993 (+ 69us) server_negotiation.cc:407] Connection header received
0923 15:05:48.019073 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:05:48.019073 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:05:48.019077 (+ 4us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:05:48.019117 (+ 40us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:05:48.019717 (+ 600us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:05:48.019718 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:05:48.019718 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 15:05:48.019722 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:05:48.019736 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 15:05:48.019740 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:05:48.019787 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0923 15:05:48.019887 (+ 100us) server_negotiation.cc:299] Negotiation successful
0923 15:05:48.019899 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":40}
I20260923 15:05:48.020375 1637 raft_consensus.cc:504] T 00000000000000000000000000000000 P b2e62979304d48759f47641d4ce01bb8 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2e62979304d48759f47641d4ce01bb8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37147 } }
I20260923 15:05:48.020493 1637 leader_election.cc:302] T 00000000000000000000000000000000 P b2e62979304d48759f47641d4ce01bb8 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: b2e62979304d48759f47641d4ce01bb8; no voters:
I20260923 15:05:48.020670 1637 leader_election.cc:288] T 00000000000000000000000000000000 P b2e62979304d48759f47641d4ce01bb8 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 15:05:48.020699 1647 raft_consensus.cc:2781] T 00000000000000000000000000000000 P b2e62979304d48759f47641d4ce01bb8 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 15:05:48.020843 1647 raft_consensus.cc:686] T 00000000000000000000000000000000 P b2e62979304d48759f47641d4ce01bb8 [term 1 LEADER]: Becoming Leader. State: Replica: b2e62979304d48759f47641d4ce01bb8, State: Running, Role: LEADER
I20260923 15:05:48.020928 1647 consensus_queue.cc:237] T 00000000000000000000000000000000 P b2e62979304d48759f47641d4ce01bb8 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2e62979304d48759f47641d4ce01bb8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37147 } }
I20260923 15:05:48.020929 1637 sys_catalog.cc:564] T 00000000000000000000000000000000 P b2e62979304d48759f47641d4ce01bb8 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 15:05:48.021302 1649 sys_catalog.cc:455] T 00000000000000000000000000000000 P b2e62979304d48759f47641d4ce01bb8 [sys.catalog]: SysCatalogTable state changed. Reason: New leader b2e62979304d48759f47641d4ce01bb8. Latest consensus state: current_term: 1 leader_uuid: "b2e62979304d48759f47641d4ce01bb8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2e62979304d48759f47641d4ce01bb8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37147 } } }
I20260923 15:05:48.021382 1649 sys_catalog.cc:458] T 00000000000000000000000000000000 P b2e62979304d48759f47641d4ce01bb8 [sys.catalog]: This master's current role is: LEADER
I20260923 15:05:48.021332 1648 sys_catalog.cc:455] T 00000000000000000000000000000000 P b2e62979304d48759f47641d4ce01bb8 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "b2e62979304d48759f47641d4ce01bb8" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "b2e62979304d48759f47641d4ce01bb8" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 37147 } } }
I20260923 15:05:48.021557 1648 sys_catalog.cc:458] T 00000000000000000000000000000000 P b2e62979304d48759f47641d4ce01bb8 [sys.catalog]: This master's current role is: LEADER
I20260923 15:05:48.022212 1653 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 15:05:48.022696 1653 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 15:05:48.024276 1653 catalog_manager.cc:1347] Generated new cluster ID: 98507381bc6a4a4b9f6285b58fb6c4f3
I20260923 15:05:48.024338 1653 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 15:05:48.045691 1653 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 15:05:48.045924 1653 catalog_manager.cc:1504] Loading token signing keys...
I20260923 15:05:48.049439 1653 catalog_manager.cc:5868] T 00000000000000000000000000000000 P b2e62979304d48759f47641d4ce01bb8: Generated new TSK 0
I20260923 15:05:48.049693 1653 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 15:05:48.060101 13614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKC4soy/build/release/bin/kudu with pid 1569
2026-09-23T15:05:48Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/2 (1350 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/3
Loading random data
Initializing database '/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790175691491353-13614-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 15:05:48 dist-test-slave-cnb0 krb5kdc[1668](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 15:05:48 dist-test-slave-cnb0 krb5kdc[1668](info): set up 2 sockets
Sep 23 15:05:48 dist-test-slave-cnb0 krb5kdc[1668](info): commencing operation
krb5kdc: starting...
W20260923 15:05:49.160161 13614 mini_kdc.cc:121] Time spent starting KDC: real 1.082s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790175691491353-13614-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790175691491353-13614-0/krb5kdc/test-user.keytab.
Sep 23 15:05:49 dist-test-slave-cnb0 krb5kdc[1668](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175949, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T15:05:49Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T15:05:49Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790175691491353-13614-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790175691491353-13614-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260923 15:05:49.291257 13614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKC4soy/build/release/bin/kudu
/tmp/dist-test-taskKC4soy/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790175691491353-13614-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790175691491353-13614-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790175691491353-13614-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790175691491353-13614-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42817
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790175691491353-13614-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:39651
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:42817
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790175691491353-13614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790175691491353-13614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790175691491353-13614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:05:49.372604 1681 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:05:49.372872 1681 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:05:49.372911 1681 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:05:49.374120 1681 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 15:05:49.374197 1681 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:05:49.374221 1681 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:05:49.374243 1681 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 15:05:49.374274 1681 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 15:05:49.375835 1681 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:39651
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790175691491353-13614-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790175691491353-13614-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:42817
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790175691491353-13614-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:42817
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790175691491353-13614-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1681
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790175691491353-13614-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:05:49.376144 1681 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:05:49.376382 1681 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:05:49.381981 1687 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:05:49.381987 1689 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:05:49.382292 1681 server_base.cc:1034] running on GCE node
W20260923 15:05:49.382000 1686 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:05:49.382614 1681 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:05:49.382979 1681 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:05:49.384150 1681 hybrid_clock.cc:648] HybridClock initialized: now 1790175949384140 us; error 42 us; skew 500 ppm
Sep 23 15:05:49 dist-test-slave-cnb0 krb5kdc[1668](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175949, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:05:49.386795 1681 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260923 15:05:49.387305 1681 webserver.cc:460] Webserver started at http://127.0.0.1:46573/ using document root <none> and password file <none>
I20260923 15:05:49.387557 1681 fs_manager.cc:359] Metadata directory not provided
I20260923 15:05:49.387635 1681 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:05:49.387784 1681 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:05:49.388885 1681 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790175691491353-13614-0/minicluster-data/master-0/data/instance:
uuid: "fc89abd26cf341a6a3ec6f513aa0835f"
format_stamp: "Formatted at 2026-09-23 15:05:49 on dist-test-slave-cnb0"
server_key: "32a826ac3f7de6b68498c22e9a2ebf80"
server_key_iv: "4e2878215cd22506fcdb0c9f07adf304"
server_key_version: "encryptionkey@0"
I20260923 15:05:49.389261 1681 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790175691491353-13614-0/minicluster-data/master-0/wal/instance:
uuid: "fc89abd26cf341a6a3ec6f513aa0835f"
format_stamp: "Formatted at 2026-09-23 15:05:49 on dist-test-slave-cnb0"
server_key: "32a826ac3f7de6b68498c22e9a2ebf80"
server_key_iv: "4e2878215cd22506fcdb0c9f07adf304"
server_key_version: "encryptionkey@0"
I20260923 15:05:49.390697 1681 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 15:05:49.391589 1696 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:05:49.391809 1681 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260923 15:05:49.391921 1681 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790175691491353-13614-0/minicluster-data/master-0/data,/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790175691491353-13614-0/minicluster-data/master-0/wal
uuid: "fc89abd26cf341a6a3ec6f513aa0835f"
format_stamp: "Formatted at 2026-09-23 15:05:49 on dist-test-slave-cnb0"
server_key: "32a826ac3f7de6b68498c22e9a2ebf80"
server_key_iv: "4e2878215cd22506fcdb0c9f07adf304"
server_key_version: "encryptionkey@0"
I20260923 15:05:49.392007 1681 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790175691491353-13614-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790175691491353-13614-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790175691491353-13614-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:05:49.406239 1681 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:05:49.408463 1681 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:05:49.408638 1681 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:05:49.412804 1681 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:42817
I20260923 15:05:49.412843 1748 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:42817 every 8 connection(s)
I20260923 15:05:49.413225 1681 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790175691491353-13614-0/minicluster-data/master-0/data/info.pb
I20260923 15:05:49.414188 1749 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:05:49.416456 13614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKC4soy/build/release/bin/kudu as pid 1681
I20260923 15:05:49.416579 13614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_3.1790175691491353-13614-0/minicluster-data/master-0/wal/instance
I20260923 15:05:49.416774 13614 external_mini_cluster.cc:1428] Setting key 18820c861557cc9caeb2e804b00495aa
I20260923 15:05:49.417043 1749 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fc89abd26cf341a6a3ec6f513aa0835f: Bootstrap starting.
I20260923 15:05:49.417778 1749 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P fc89abd26cf341a6a3ec6f513aa0835f: Neither blocks nor log segments found. Creating new log.
I20260923 15:05:49.418071 1749 log.cc:824] T 00000000000000000000000000000000 P fc89abd26cf341a6a3ec6f513aa0835f: Log is configured to *not* fsync() on all Append() calls
I20260923 15:05:49.418757 1749 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P fc89abd26cf341a6a3ec6f513aa0835f: No bootstrap required, opened a new log
I20260923 15:05:49.420606 1749 raft_consensus.cc:348] T 00000000000000000000000000000000 P fc89abd26cf341a6a3ec6f513aa0835f [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc89abd26cf341a6a3ec6f513aa0835f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42817 } }
I20260923 15:05:49.420771 1749 raft_consensus.cc:374] T 00000000000000000000000000000000 P fc89abd26cf341a6a3ec6f513aa0835f [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:05:49.420820 1749 raft_consensus.cc:729] T 00000000000000000000000000000000 P fc89abd26cf341a6a3ec6f513aa0835f [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: fc89abd26cf341a6a3ec6f513aa0835f, State: Initialized, Role: FOLLOWER
I20260923 15:05:49.420967 1749 consensus_queue.cc:260] T 00000000000000000000000000000000 P fc89abd26cf341a6a3ec6f513aa0835f [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc89abd26cf341a6a3ec6f513aa0835f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42817 } }
I20260923 15:05:49.421075 1749 raft_consensus.cc:388] T 00000000000000000000000000000000 P fc89abd26cf341a6a3ec6f513aa0835f [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 15:05:49.421119 1749 raft_consensus.cc:482] T 00000000000000000000000000000000 P fc89abd26cf341a6a3ec6f513aa0835f [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 15:05:49.421168 1749 raft_consensus.cc:3037] T 00000000000000000000000000000000 P fc89abd26cf341a6a3ec6f513aa0835f [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:05:49.421810 1749 raft_consensus.cc:504] T 00000000000000000000000000000000 P fc89abd26cf341a6a3ec6f513aa0835f [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc89abd26cf341a6a3ec6f513aa0835f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42817 } }
I20260923 15:05:49.421929 1749 leader_election.cc:302] T 00000000000000000000000000000000 P fc89abd26cf341a6a3ec6f513aa0835f [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: fc89abd26cf341a6a3ec6f513aa0835f; no voters:
I20260923 15:05:49.422117 1749 leader_election.cc:288] T 00000000000000000000000000000000 P fc89abd26cf341a6a3ec6f513aa0835f [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 15:05:49.422135 1754 raft_consensus.cc:2781] T 00000000000000000000000000000000 P fc89abd26cf341a6a3ec6f513aa0835f [term 1 FOLLOWER]: Leader election won for term 1
I20260923 15:05:49.422272 1754 raft_consensus.cc:686] T 00000000000000000000000000000000 P fc89abd26cf341a6a3ec6f513aa0835f [term 1 LEADER]: Becoming Leader. State: Replica: fc89abd26cf341a6a3ec6f513aa0835f, State: Running, Role: LEADER
I20260923 15:05:49.422355 1754 consensus_queue.cc:237] T 00000000000000000000000000000000 P fc89abd26cf341a6a3ec6f513aa0835f [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc89abd26cf341a6a3ec6f513aa0835f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42817 } }
I20260923 15:05:49.422420 1749 sys_catalog.cc:564] T 00000000000000000000000000000000 P fc89abd26cf341a6a3ec6f513aa0835f [sys.catalog]: configured and running, proceeding with master startup.
I20260923 15:05:49.422722 1756 sys_catalog.cc:455] T 00000000000000000000000000000000 P fc89abd26cf341a6a3ec6f513aa0835f [sys.catalog]: SysCatalogTable state changed. Reason: New leader fc89abd26cf341a6a3ec6f513aa0835f. Latest consensus state: current_term: 1 leader_uuid: "fc89abd26cf341a6a3ec6f513aa0835f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc89abd26cf341a6a3ec6f513aa0835f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42817 } } }
I20260923 15:05:49.422838 1756 sys_catalog.cc:458] T 00000000000000000000000000000000 P fc89abd26cf341a6a3ec6f513aa0835f [sys.catalog]: This master's current role is: LEADER
I20260923 15:05:49.422712 1755 sys_catalog.cc:455] T 00000000000000000000000000000000 P fc89abd26cf341a6a3ec6f513aa0835f [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "fc89abd26cf341a6a3ec6f513aa0835f" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "fc89abd26cf341a6a3ec6f513aa0835f" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 42817 } } }
I20260923 15:05:49.423206 1755 sys_catalog.cc:458] T 00000000000000000000000000000000 P fc89abd26cf341a6a3ec6f513aa0835f [sys.catalog]: This master's current role is: LEADER
I20260923 15:05:49.423412 1760 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
Sep 23 15:05:49 dist-test-slave-cnb0 krb5kdc[1668](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175949, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260923 15:05:49.424021 1760 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 15:05:49.425962 1760 catalog_manager.cc:1347] Generated new cluster ID: 23e1c41876754991aa22875ddeeb9f97
I20260923 15:05:49.426018 1760 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 15:05:49.428380 1752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:49.417372 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49177 (local address 127.0.0.1:42817)
0923 15:05:49.417559 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0923 15:05:49.417569 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0923 15:05:49.417607 (+ 38us) server_negotiation.cc:407] Connection header received
0923 15:05:49.418093 (+ 486us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:05:49.418100 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:05:49.418188 (+ 88us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:05:49.418338 (+ 150us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:05:49.419038 (+ 700us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:05:49.419914 (+ 876us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:05:49.420699 (+ 785us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:05:49.421290 (+ 591us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:05:49.421415 (+ 125us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:05:49.424488 (+ 3073us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:05:49.424492 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:05:49.424492 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:05:49.424505 (+ 13us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:05:49.426640 (+ 2135us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:05:49.427114 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:05:49.427118 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:05:49.427118 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:05:49.427229 (+ 111us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:05:49.427462 (+ 233us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:05:49.427462 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:05:49.427463 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:05:49.427758 (+ 295us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:05:49.427900 (+ 142us) server_negotiation.cc:1035] Waiting for connection context
0923 15:05:49.428030 (+ 130us) server_negotiation.cc:299] Negotiation successful
0923 15:05:49.428175 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":136,"thread_start_us":97,"threads_started":1}
I20260923 15:05:49.431803 13614 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 15:05:49.440567 1752 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:49.433399 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34673 (local address 127.0.0.1:42817)
0923 15:05:49.433453 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0923 15:05:49.433455 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:05:49.433580 (+ 125us) server_negotiation.cc:407] Connection header received
0923 15:05:49.433666 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:05:49.433667 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:05:49.433677 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:05:49.433704 (+ 27us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:05:49.434352 (+ 648us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:05:49.435104 (+ 752us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:05:49.436082 (+ 978us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:05:49.436420 (+ 338us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:05:49.436456 (+ 36us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:05:49.437888 (+ 1432us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:05:49.437890 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:05:49.437890 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:05:49.437897 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:05:49.439459 (+ 1562us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:05:49.439821 (+ 362us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:05:49.439822 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:05:49.439823 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:05:49.439872 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:05:49.440037 (+ 165us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:05:49.440037 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:05:49.440037 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:05:49.440148 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:05:49.440222 (+ 74us) server_negotiation.cc:1035] Waiting for connection context
0923 15:05:49.440406 (+ 184us) server_negotiation.cc:299] Negotiation successful
0923 15:05:49.440474 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260923 15:05:49.448148 1760 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 15:05:49.448697 1760 catalog_manager.cc:1504] Loading token signing keys...
I20260923 15:05:49.457399 1760 catalog_manager.cc:5868] T 00000000000000000000000000000000 P fc89abd26cf341a6a3ec6f513aa0835f: Generated new TSK 0
I20260923 15:05:49.457633 1760 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 15:05:49.462827 13614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKC4soy/build/release/bin/kudu with pid 1681
2026-09-23T15:05:49Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/3 (1402 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/4
Loading random data
Initializing database '/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790175691491353-13614-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 15:05:49 dist-test-slave-cnb0 krb5kdc[1780](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 15:05:49 dist-test-slave-cnb0 krb5kdc[1780](info): set up 2 sockets
Sep 23 15:05:49 dist-test-slave-cnb0 krb5kdc[1780](info): commencing operation
krb5kdc: starting...
W20260923 15:05:50.544203 13614 mini_kdc.cc:121] Time spent starting KDC: real 1.063s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790175691491353-13614-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790175691491353-13614-0/krb5kdc/test-user.keytab.
Sep 23 15:05:50 dist-test-slave-cnb0 krb5kdc[1780](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175950, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T15:05:50Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T15:05:50Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790175691491353-13614-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790175691491353-13614-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260923 15:05:50.675062 13614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKC4soy/build/release/bin/kudu
/tmp/dist-test-taskKC4soy/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790175691491353-13614-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790175691491353-13614-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790175691491353-13614-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790175691491353-13614-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34787
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790175691491353-13614-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:40691
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:34787
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790175691491353-13614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790175691491353-13614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790175691491353-13614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:05:50.759654 1793 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:05:50.759917 1793 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:05:50.759948 1793 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:05:50.761190 1793 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 15:05:50.761235 1793 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:05:50.761250 1793 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:05:50.761262 1793 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 15:05:50.761276 1793 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 15:05:50.762852 1793 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:40691
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790175691491353-13614-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790175691491353-13614-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:34787
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790175691491353-13614-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:34787
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790175691491353-13614-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1793
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790175691491353-13614-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:05:50.763118 1793 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:05:50.763310 1793 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:05:50.768863 1798 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:05:50.768905 1801 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:05:50.768893 1799 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:05:50.769351 1793 server_base.cc:1034] running on GCE node
I20260923 15:05:50.769655 1793 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:05:50.770020 1793 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:05:50.771204 1793 hybrid_clock.cc:648] HybridClock initialized: now 1790175950771180 us; error 42 us; skew 500 ppm
Sep 23 15:05:50 dist-test-slave-cnb0 krb5kdc[1780](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175950, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:05:50.778081 1793 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260923 15:05:50.778708 1793 webserver.cc:460] Webserver started at http://127.0.0.1:44005/ using document root <none> and password file <none>
I20260923 15:05:50.779072 1793 fs_manager.cc:359] Metadata directory not provided
I20260923 15:05:50.779155 1793 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:05:50.779341 1793 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:05:50.780668 1793 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790175691491353-13614-0/minicluster-data/master-0/data/instance:
uuid: "57da84d897254541b565f78a8c6e8553"
format_stamp: "Formatted at 2026-09-23 15:05:50 on dist-test-slave-cnb0"
server_key: "c8a84387f743053597e93ff6182694cf"
server_key_iv: "dae5bfa77db982faba004e2f9876fa1d"
server_key_version: "encryptionkey@0"
I20260923 15:05:50.781145 1793 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790175691491353-13614-0/minicluster-data/master-0/wal/instance:
uuid: "57da84d897254541b565f78a8c6e8553"
format_stamp: "Formatted at 2026-09-23 15:05:50 on dist-test-slave-cnb0"
server_key: "c8a84387f743053597e93ff6182694cf"
server_key_iv: "dae5bfa77db982faba004e2f9876fa1d"
server_key_version: "encryptionkey@0"
I20260923 15:05:50.782796 1793 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 15:05:50.783740 1808 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:05:50.784024 1793 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:05:50.784161 1793 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790175691491353-13614-0/minicluster-data/master-0/data,/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790175691491353-13614-0/minicluster-data/master-0/wal
uuid: "57da84d897254541b565f78a8c6e8553"
format_stamp: "Formatted at 2026-09-23 15:05:50 on dist-test-slave-cnb0"
server_key: "c8a84387f743053597e93ff6182694cf"
server_key_iv: "dae5bfa77db982faba004e2f9876fa1d"
server_key_version: "encryptionkey@0"
I20260923 15:05:50.784248 1793 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790175691491353-13614-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790175691491353-13614-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790175691491353-13614-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:05:50.795946 1793 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:05:50.798256 1793 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:05:50.798444 1793 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:05:50.803507 1793 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:34787
I20260923 15:05:50.803547 1860 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:34787 every 8 connection(s)
I20260923 15:05:50.803966 1793 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790175691491353-13614-0/minicluster-data/master-0/data/info.pb
I20260923 15:05:50.805011 1861 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:05:50.807860 1861 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 57da84d897254541b565f78a8c6e8553: Bootstrap starting.
I20260923 15:05:50.808502 1861 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 57da84d897254541b565f78a8c6e8553: Neither blocks nor log segments found. Creating new log.
I20260923 15:05:50.808785 1861 log.cc:824] T 00000000000000000000000000000000 P 57da84d897254541b565f78a8c6e8553: Log is configured to *not* fsync() on all Append() calls
I20260923 15:05:50.809682 1861 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 57da84d897254541b565f78a8c6e8553: No bootstrap required, opened a new log
I20260923 15:05:50.810506 13614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKC4soy/build/release/bin/kudu as pid 1793
I20260923 15:05:50.810683 13614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_4.1790175691491353-13614-0/minicluster-data/master-0/wal/instance
I20260923 15:05:50.810940 13614 external_mini_cluster.cc:1428] Setting key e28269addd692f1fbdc315dc320cbee5
I20260923 15:05:50.812508 1861 raft_consensus.cc:348] T 00000000000000000000000000000000 P 57da84d897254541b565f78a8c6e8553 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57da84d897254541b565f78a8c6e8553" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34787 } }
I20260923 15:05:50.812731 1861 raft_consensus.cc:374] T 00000000000000000000000000000000 P 57da84d897254541b565f78a8c6e8553 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:05:50.812827 1861 raft_consensus.cc:729] T 00000000000000000000000000000000 P 57da84d897254541b565f78a8c6e8553 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 57da84d897254541b565f78a8c6e8553, State: Initialized, Role: FOLLOWER
I20260923 15:05:50.813024 1861 consensus_queue.cc:260] T 00000000000000000000000000000000 P 57da84d897254541b565f78a8c6e8553 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57da84d897254541b565f78a8c6e8553" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34787 } }
I20260923 15:05:50.813153 1861 raft_consensus.cc:388] T 00000000000000000000000000000000 P 57da84d897254541b565f78a8c6e8553 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 15:05:50.813199 1861 raft_consensus.cc:482] T 00000000000000000000000000000000 P 57da84d897254541b565f78a8c6e8553 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 15:05:50.813251 1861 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 57da84d897254541b565f78a8c6e8553 [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:05:50.814235 1861 raft_consensus.cc:504] T 00000000000000000000000000000000 P 57da84d897254541b565f78a8c6e8553 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57da84d897254541b565f78a8c6e8553" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34787 } }
I20260923 15:05:50.814365 1861 leader_election.cc:302] T 00000000000000000000000000000000 P 57da84d897254541b565f78a8c6e8553 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 57da84d897254541b565f78a8c6e8553; no voters:
I20260923 15:05:50.814568 1861 leader_election.cc:288] T 00000000000000000000000000000000 P 57da84d897254541b565f78a8c6e8553 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 15:05:50.814709 1866 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 57da84d897254541b565f78a8c6e8553 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 15:05:50.814841 1861 sys_catalog.cc:564] T 00000000000000000000000000000000 P 57da84d897254541b565f78a8c6e8553 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 15:05:50.814910 1866 raft_consensus.cc:686] T 00000000000000000000000000000000 P 57da84d897254541b565f78a8c6e8553 [term 1 LEADER]: Becoming Leader. State: Replica: 57da84d897254541b565f78a8c6e8553, State: Running, Role: LEADER
I20260923 15:05:50.815045 1866 consensus_queue.cc:237] T 00000000000000000000000000000000 P 57da84d897254541b565f78a8c6e8553 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57da84d897254541b565f78a8c6e8553" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34787 } }
I20260923 15:05:50.815476 1867 sys_catalog.cc:455] T 00000000000000000000000000000000 P 57da84d897254541b565f78a8c6e8553 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 57da84d897254541b565f78a8c6e8553. Latest consensus state: current_term: 1 leader_uuid: "57da84d897254541b565f78a8c6e8553" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57da84d897254541b565f78a8c6e8553" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34787 } } }
I20260923 15:05:50.815678 1867 sys_catalog.cc:458] T 00000000000000000000000000000000 P 57da84d897254541b565f78a8c6e8553 [sys.catalog]: This master's current role is: LEADER
I20260923 15:05:50.816354 1868 sys_catalog.cc:455] T 00000000000000000000000000000000 P 57da84d897254541b565f78a8c6e8553 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "57da84d897254541b565f78a8c6e8553" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "57da84d897254541b565f78a8c6e8553" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 34787 } } }
I20260923 15:05:50.816501 1868 sys_catalog.cc:458] T 00000000000000000000000000000000 P 57da84d897254541b565f78a8c6e8553 [sys.catalog]: This master's current role is: LEADER
I20260923 15:05:50.816413 1875 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 15:05:50.817384 1875 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 15:05:50.819105 1865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:50.811587 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41917 (local address 127.0.0.1:34787)
0923 15:05:50.811827 (+ 240us) server_negotiation.cc:206] Beginning negotiation
0923 15:05:50.811839 (+ 12us) server_negotiation.cc:399] Waiting for connection header
0923 15:05:50.811888 (+ 49us) server_negotiation.cc:407] Connection header received
0923 15:05:50.812455 (+ 567us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:05:50.812466 (+ 11us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:05:50.812551 (+ 85us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:05:50.812732 (+ 181us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:05:50.813001 (+ 269us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:05:50.815001 (+ 2000us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:05:50.816283 (+ 1282us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:05:50.816894 (+ 611us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:05:50.817006 (+ 112us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:05:50.817949 (+ 943us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:05:50.817953 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:05:50.817954 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 15:05:50.818257 (+ 303us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:05:50.818303 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 15:05:50.818320 (+ 17us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:05:50.818419 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0923 15:05:50.818707 (+ 288us) server_negotiation.cc:299] Negotiation successful
0923 15:05:50.818792 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":166,"thread_start_us":97,"threads_started":1}
I20260923 15:05:50.819506 1875 catalog_manager.cc:1347] Generated new cluster ID: 51b251017cdf44c39402f4933381e75e
I20260923 15:05:50.819571 1875 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 15:05:50.820217 13614 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 15:05:50.828860 1865 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:50.825640 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38593 (local address 127.0.0.1:34787)
0923 15:05:50.825685 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0923 15:05:50.825686 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 15:05:50.825858 (+ 172us) server_negotiation.cc:407] Connection header received
0923 15:05:50.825942 (+ 84us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:05:50.825943 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:05:50.825955 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:05:50.826039 (+ 84us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:05:50.826210 (+ 171us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:05:50.826780 (+ 570us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:05:50.827568 (+ 788us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:05:50.827781 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:05:50.827866 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:05:50.828592 (+ 726us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:05:50.828594 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:05:50.828594 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 15:05:50.828599 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:05:50.828615 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 15:05:50.828619 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:05:50.828672 (+ 53us) server_negotiation.cc:1035] Waiting for connection context
0923 15:05:50.828738 (+ 66us) server_negotiation.cc:299] Negotiation successful
0923 15:05:50.828766 (+ 28us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260923 15:05:50.845191 1875 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 15:05:50.845791 1875 catalog_manager.cc:1504] Loading token signing keys...
I20260923 15:05:50.850293 1875 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 57da84d897254541b565f78a8c6e8553: Generated new TSK 0
I20260923 15:05:50.850517 1875 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 15:05:50.853296 13614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKC4soy/build/release/bin/kudu with pid 1793
2026-09-23T15:05:50Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/4 (1391 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/5
Loading random data
Initializing database '/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790175691491353-13614-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 15:05:50 dist-test-slave-cnb0 krb5kdc[1892](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 15:05:50 dist-test-slave-cnb0 krb5kdc[1892](info): set up 2 sockets
Sep 23 15:05:50 dist-test-slave-cnb0 krb5kdc[1892](info): commencing operation
krb5kdc: starting...
W20260923 15:05:51.896214 13614 mini_kdc.cc:121] Time spent starting KDC: real 1.024s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790175691491353-13614-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790175691491353-13614-0/krb5kdc/test-user.keytab.
Sep 23 15:05:51 dist-test-slave-cnb0 krb5kdc[1892](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175951, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T15:05:51Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T15:05:51Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790175691491353-13614-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790175691491353-13614-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260923 15:05:52.024257 13614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKC4soy/build/release/bin/kudu
/tmp/dist-test-taskKC4soy/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790175691491353-13614-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790175691491353-13614-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790175691491353-13614-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790175691491353-13614-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39533
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790175691491353-13614-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:37909
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39533
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790175691491353-13614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790175691491353-13614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790175691491353-13614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:05:52.106472 1905 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:05:52.106746 1905 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:05:52.106791 1905 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:05:52.108031 1905 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 15:05:52.108114 1905 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:05:52.108142 1905 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:05:52.108174 1905 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 15:05:52.108199 1905 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 15:05:52.109761 1905 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:37909
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790175691491353-13614-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790175691491353-13614-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39533
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790175691491353-13614-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39533
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790175691491353-13614-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.1905
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790175691491353-13614-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:05:52.110055 1905 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:05:52.110292 1905 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:05:52.115566 1910 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:05:52.115557 1911 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:05:52.115587 1913 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:05:52.115996 1905 server_base.cc:1034] running on GCE node
I20260923 15:05:52.116338 1905 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:05:52.116676 1905 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:05:52.117853 1905 hybrid_clock.cc:648] HybridClock initialized: now 1790175952117840 us; error 35 us; skew 500 ppm
Sep 23 15:05:52 dist-test-slave-cnb0 krb5kdc[1892](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175952, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:05:52.120704 1905 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260923 15:05:52.121248 1905 webserver.cc:460] Webserver started at http://127.0.0.1:37721/ using document root <none> and password file <none>
I20260923 15:05:52.121521 1905 fs_manager.cc:359] Metadata directory not provided
I20260923 15:05:52.121572 1905 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:05:52.121716 1905 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:05:52.122856 1905 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790175691491353-13614-0/minicluster-data/master-0/data/instance:
uuid: "6eef998addd84aeea362d6ac79e6d85a"
format_stamp: "Formatted at 2026-09-23 15:05:52 on dist-test-slave-cnb0"
server_key: "86856a88203ecb20079c7f7e9bef9bfa"
server_key_iv: "561a9c37c2823d4570ff7ab09a00b926"
server_key_version: "encryptionkey@0"
I20260923 15:05:52.123327 1905 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790175691491353-13614-0/minicluster-data/master-0/wal/instance:
uuid: "6eef998addd84aeea362d6ac79e6d85a"
format_stamp: "Formatted at 2026-09-23 15:05:52 on dist-test-slave-cnb0"
server_key: "86856a88203ecb20079c7f7e9bef9bfa"
server_key_iv: "561a9c37c2823d4570ff7ab09a00b926"
server_key_version: "encryptionkey@0"
I20260923 15:05:52.124869 1905 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 15:05:52.125877 1920 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:05:52.126142 1905 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:05:52.126276 1905 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790175691491353-13614-0/minicluster-data/master-0/data,/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790175691491353-13614-0/minicluster-data/master-0/wal
uuid: "6eef998addd84aeea362d6ac79e6d85a"
format_stamp: "Formatted at 2026-09-23 15:05:52 on dist-test-slave-cnb0"
server_key: "86856a88203ecb20079c7f7e9bef9bfa"
server_key_iv: "561a9c37c2823d4570ff7ab09a00b926"
server_key_version: "encryptionkey@0"
I20260923 15:05:52.126369 1905 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790175691491353-13614-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790175691491353-13614-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790175691491353-13614-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:05:52.130154 1905 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:05:52.133093 1905 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:05:52.133338 1905 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:05:52.137835 1905 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39533
I20260923 15:05:52.137871 1972 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39533 every 8 connection(s)
I20260923 15:05:52.138244 1905 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790175691491353-13614-0/minicluster-data/master-0/data/info.pb
I20260923 15:05:52.139307 1973 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:05:52.139394 13614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKC4soy/build/release/bin/kudu as pid 1905
I20260923 15:05:52.139518 13614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_5.1790175691491353-13614-0/minicluster-data/master-0/wal/instance
I20260923 15:05:52.139717 13614 external_mini_cluster.cc:1428] Setting key acaf40a20a14e10a2db65554b1c5b1d0
I20260923 15:05:52.142333 1976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:52.140318 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44187 (local address 127.0.0.1:39533)
0923 15:05:52.140510 (+ 192us) server_negotiation.cc:206] Beginning negotiation
0923 15:05:52.140518 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0923 15:05:52.140545 (+ 27us) server_negotiation.cc:407] Connection header received
0923 15:05:52.141006 (+ 461us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:05:52.141012 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:05:52.141039 (+ 27us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:05:52.141166 (+ 127us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:05:52.141847 (+ 681us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:05:52.141851 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:05:52.141851 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 15:05:52.141854 (+ 3us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:05:52.141894 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 15:05:52.141905 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:05:52.141964 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0923 15:05:52.142053 (+ 89us) server_negotiation.cc:299] Negotiation successful
0923 15:05:52.142110 (+ 57us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":86,"threads_started":1}
I20260923 15:05:52.143287 1973 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6eef998addd84aeea362d6ac79e6d85a: Bootstrap starting.
I20260923 15:05:52.143756 13614 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 15:05:52.144101 1973 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6eef998addd84aeea362d6ac79e6d85a: Neither blocks nor log segments found. Creating new log.
I20260923 15:05:52.144439 1973 log.cc:824] T 00000000000000000000000000000000 P 6eef998addd84aeea362d6ac79e6d85a: Log is configured to *not* fsync() on all Append() calls
I20260923 15:05:52.145291 1973 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6eef998addd84aeea362d6ac79e6d85a: No bootstrap required, opened a new log
I20260923 15:05:52.146601 1976 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:52.145452 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:45461 (local address 127.0.0.1:39533)
0923 15:05:52.145508 (+ 56us) server_negotiation.cc:206] Beginning negotiation
0923 15:05:52.145510 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:05:52.145711 (+ 201us) server_negotiation.cc:407] Connection header received
0923 15:05:52.145759 (+ 48us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:05:52.145760 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:05:52.145766 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:05:52.145830 (+ 64us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:05:52.146387 (+ 557us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:05:52.146388 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:05:52.146389 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 15:05:52.146393 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:05:52.146409 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 15:05:52.146414 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:05:52.146480 (+ 66us) server_negotiation.cc:1035] Waiting for connection context
0923 15:05:52.146515 (+ 35us) server_negotiation.cc:299] Negotiation successful
0923 15:05:52.146528 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":31}
I20260923 15:05:52.147926 1973 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6eef998addd84aeea362d6ac79e6d85a [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6eef998addd84aeea362d6ac79e6d85a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39533 } }
I20260923 15:05:52.148080 1973 raft_consensus.cc:374] T 00000000000000000000000000000000 P 6eef998addd84aeea362d6ac79e6d85a [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:05:52.148103 1973 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6eef998addd84aeea362d6ac79e6d85a [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6eef998addd84aeea362d6ac79e6d85a, State: Initialized, Role: FOLLOWER
I20260923 15:05:52.148226 1973 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6eef998addd84aeea362d6ac79e6d85a [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6eef998addd84aeea362d6ac79e6d85a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39533 } }
I20260923 15:05:52.148316 1973 raft_consensus.cc:388] T 00000000000000000000000000000000 P 6eef998addd84aeea362d6ac79e6d85a [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 15:05:52.148340 1973 raft_consensus.cc:482] T 00000000000000000000000000000000 P 6eef998addd84aeea362d6ac79e6d85a [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 15:05:52.148376 1973 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6eef998addd84aeea362d6ac79e6d85a [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:05:52.149161 1973 raft_consensus.cc:504] T 00000000000000000000000000000000 P 6eef998addd84aeea362d6ac79e6d85a [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6eef998addd84aeea362d6ac79e6d85a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39533 } }
I20260923 15:05:52.149298 1973 leader_election.cc:302] T 00000000000000000000000000000000 P 6eef998addd84aeea362d6ac79e6d85a [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 6eef998addd84aeea362d6ac79e6d85a; no voters:
I20260923 15:05:52.149497 1973 leader_election.cc:288] T 00000000000000000000000000000000 P 6eef998addd84aeea362d6ac79e6d85a [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 15:05:52.149811 1984 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 6eef998addd84aeea362d6ac79e6d85a [term 1 FOLLOWER]: Leader election won for term 1
I20260923 15:05:52.149937 1973 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6eef998addd84aeea362d6ac79e6d85a [sys.catalog]: configured and running, proceeding with master startup.
I20260923 15:05:52.149976 1984 raft_consensus.cc:686] T 00000000000000000000000000000000 P 6eef998addd84aeea362d6ac79e6d85a [term 1 LEADER]: Becoming Leader. State: Replica: 6eef998addd84aeea362d6ac79e6d85a, State: Running, Role: LEADER
I20260923 15:05:52.150100 1984 consensus_queue.cc:237] T 00000000000000000000000000000000 P 6eef998addd84aeea362d6ac79e6d85a [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6eef998addd84aeea362d6ac79e6d85a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39533 } }
I20260923 15:05:52.150592 1985 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6eef998addd84aeea362d6ac79e6d85a [sys.catalog]: SysCatalogTable state changed. Reason: New leader 6eef998addd84aeea362d6ac79e6d85a. Latest consensus state: current_term: 1 leader_uuid: "6eef998addd84aeea362d6ac79e6d85a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6eef998addd84aeea362d6ac79e6d85a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39533 } } }
I20260923 15:05:52.150713 1985 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6eef998addd84aeea362d6ac79e6d85a [sys.catalog]: This master's current role is: LEADER
I20260923 15:05:52.150595 1983 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6eef998addd84aeea362d6ac79e6d85a [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "6eef998addd84aeea362d6ac79e6d85a" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "6eef998addd84aeea362d6ac79e6d85a" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39533 } } }
I20260923 15:05:52.151022 1983 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6eef998addd84aeea362d6ac79e6d85a [sys.catalog]: This master's current role is: LEADER
I20260923 15:05:52.151026 1992 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 15:05:52.151665 1992 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 15:05:52.153235 1992 catalog_manager.cc:1347] Generated new cluster ID: 6e069a5046b64cf1b8b398510de8f0ae
I20260923 15:05:52.153303 1992 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 15:05:52.173293 1992 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 15:05:52.173506 1992 catalog_manager.cc:1504] Loading token signing keys...
I20260923 15:05:52.180352 1992 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 6eef998addd84aeea362d6ac79e6d85a: Generated new TSK 0
I20260923 15:05:52.180603 1992 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 15:05:52.186175 13614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKC4soy/build/release/bin/kudu with pid 1905
2026-09-23T15:05:52Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/5 (1332 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/6
Loading random data
Initializing database '/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790175691491353-13614-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 15:05:52 dist-test-slave-cnb0 krb5kdc[2004](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 15:05:52 dist-test-slave-cnb0 krb5kdc[2004](info): set up 2 sockets
Sep 23 15:05:52 dist-test-slave-cnb0 krb5kdc[2004](info): commencing operation
krb5kdc: starting...
W20260923 15:05:53.224326 13614 mini_kdc.cc:121] Time spent starting KDC: real 1.019s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790175691491353-13614-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790175691491353-13614-0/krb5kdc/test-user.keytab.
Sep 23 15:05:53 dist-test-slave-cnb0 krb5kdc[2004](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175953, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T15:05:53Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T15:05:53Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790175691491353-13614-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790175691491353-13614-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260923 15:05:53.357939 13614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKC4soy/build/release/bin/kudu
/tmp/dist-test-taskKC4soy/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790175691491353-13614-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790175691491353-13614-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790175691491353-13614-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790175691491353-13614-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40049
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790175691491353-13614-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45489
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40049
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790175691491353-13614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790175691491353-13614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790175691491353-13614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:05:53.443758 2017 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:05:53.444036 2017 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:05:53.444082 2017 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:05:53.445405 2017 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 15:05:53.445493 2017 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:05:53.445519 2017 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:05:53.445549 2017 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 15:05:53.445575 2017 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 15:05:53.447438 2017 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45489
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790175691491353-13614-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790175691491353-13614-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40049
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790175691491353-13614-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40049
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790175691491353-13614-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2017
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790175691491353-13614-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:05:53.447736 2017 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:05:53.447983 2017 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:05:53.452967 2022 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:05:53.452975 2023 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:05:53.452978 2025 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:05:53.453251 2017 server_base.cc:1034] running on GCE node
I20260923 15:05:53.453510 2017 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:05:53.453812 2017 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:05:53.454958 2017 hybrid_clock.cc:648] HybridClock initialized: now 1790175953454948 us; error 38 us; skew 500 ppm
Sep 23 15:05:53 dist-test-slave-cnb0 krb5kdc[2004](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175953, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:05:53.457499 2017 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260923 15:05:53.457963 2017 webserver.cc:460] Webserver started at http://127.0.0.1:37237/ using document root <none> and password file <none>
I20260923 15:05:53.458220 2017 fs_manager.cc:359] Metadata directory not provided
I20260923 15:05:53.458290 2017 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:05:53.458439 2017 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:05:53.459600 2017 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790175691491353-13614-0/minicluster-data/master-0/data/instance:
uuid: "7af3795ebc594d0e8ad20fecd637b0cc"
format_stamp: "Formatted at 2026-09-23 15:05:53 on dist-test-slave-cnb0"
server_key: "994fe992c6df4766187f76dd51d5fbb9"
server_key_iv: "d9eac3ed51d602e517e7592bb5da0b2d"
server_key_version: "encryptionkey@0"
I20260923 15:05:53.459954 2017 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790175691491353-13614-0/minicluster-data/master-0/wal/instance:
uuid: "7af3795ebc594d0e8ad20fecd637b0cc"
format_stamp: "Formatted at 2026-09-23 15:05:53 on dist-test-slave-cnb0"
server_key: "994fe992c6df4766187f76dd51d5fbb9"
server_key_iv: "d9eac3ed51d602e517e7592bb5da0b2d"
server_key_version: "encryptionkey@0"
I20260923 15:05:53.461344 2017 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.000s
I20260923 15:05:53.462095 2032 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:05:53.462342 2017 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:05:53.462460 2017 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790175691491353-13614-0/minicluster-data/master-0/data,/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790175691491353-13614-0/minicluster-data/master-0/wal
uuid: "7af3795ebc594d0e8ad20fecd637b0cc"
format_stamp: "Formatted at 2026-09-23 15:05:53 on dist-test-slave-cnb0"
server_key: "994fe992c6df4766187f76dd51d5fbb9"
server_key_iv: "d9eac3ed51d602e517e7592bb5da0b2d"
server_key_version: "encryptionkey@0"
I20260923 15:05:53.462540 2017 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790175691491353-13614-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790175691491353-13614-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790175691491353-13614-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:05:53.471494 2017 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:05:53.473596 2017 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:05:53.473764 2017 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:05:53.477973 2017 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40049
I20260923 15:05:53.478006 2084 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40049 every 8 connection(s)
I20260923 15:05:53.478366 2017 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790175691491353-13614-0/minicluster-data/master-0/data/info.pb
I20260923 15:05:53.479295 2085 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:05:53.481590 2085 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7af3795ebc594d0e8ad20fecd637b0cc: Bootstrap starting.
I20260923 15:05:53.482138 2085 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 7af3795ebc594d0e8ad20fecd637b0cc: Neither blocks nor log segments found. Creating new log.
I20260923 15:05:53.482381 2085 log.cc:824] T 00000000000000000000000000000000 P 7af3795ebc594d0e8ad20fecd637b0cc: Log is configured to *not* fsync() on all Append() calls
I20260923 15:05:53.483153 2085 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 7af3795ebc594d0e8ad20fecd637b0cc: No bootstrap required, opened a new log
I20260923 15:05:53.483491 13614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKC4soy/build/release/bin/kudu as pid 2017
I20260923 15:05:53.483614 13614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_6.1790175691491353-13614-0/minicluster-data/master-0/wal/instance
I20260923 15:05:53.483806 13614 external_mini_cluster.cc:1428] Setting key b365c3b8ecf56d4c32555cf77bffd193
I20260923 15:05:53.485436 2085 raft_consensus.cc:348] T 00000000000000000000000000000000 P 7af3795ebc594d0e8ad20fecd637b0cc [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7af3795ebc594d0e8ad20fecd637b0cc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40049 } }
I20260923 15:05:53.485654 2085 raft_consensus.cc:374] T 00000000000000000000000000000000 P 7af3795ebc594d0e8ad20fecd637b0cc [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:05:53.485690 2085 raft_consensus.cc:729] T 00000000000000000000000000000000 P 7af3795ebc594d0e8ad20fecd637b0cc [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 7af3795ebc594d0e8ad20fecd637b0cc, State: Initialized, Role: FOLLOWER
I20260923 15:05:53.485810 2085 consensus_queue.cc:260] T 00000000000000000000000000000000 P 7af3795ebc594d0e8ad20fecd637b0cc [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7af3795ebc594d0e8ad20fecd637b0cc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40049 } }
I20260923 15:05:53.485900 2085 raft_consensus.cc:388] T 00000000000000000000000000000000 P 7af3795ebc594d0e8ad20fecd637b0cc [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 15:05:53.485932 2085 raft_consensus.cc:482] T 00000000000000000000000000000000 P 7af3795ebc594d0e8ad20fecd637b0cc [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 15:05:53.486019 2085 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 7af3795ebc594d0e8ad20fecd637b0cc [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:05:53.486635 2085 raft_consensus.cc:504] T 00000000000000000000000000000000 P 7af3795ebc594d0e8ad20fecd637b0cc [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7af3795ebc594d0e8ad20fecd637b0cc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40049 } }
I20260923 15:05:53.486745 2085 leader_election.cc:302] T 00000000000000000000000000000000 P 7af3795ebc594d0e8ad20fecd637b0cc [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 7af3795ebc594d0e8ad20fecd637b0cc; no voters:
I20260923 15:05:53.486914 2085 leader_election.cc:288] T 00000000000000000000000000000000 P 7af3795ebc594d0e8ad20fecd637b0cc [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 15:05:53.486969 2090 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 7af3795ebc594d0e8ad20fecd637b0cc [term 1 FOLLOWER]: Leader election won for term 1
I20260923 15:05:53.487114 2090 raft_consensus.cc:686] T 00000000000000000000000000000000 P 7af3795ebc594d0e8ad20fecd637b0cc [term 1 LEADER]: Becoming Leader. State: Replica: 7af3795ebc594d0e8ad20fecd637b0cc, State: Running, Role: LEADER
I20260923 15:05:53.487176 2085 sys_catalog.cc:564] T 00000000000000000000000000000000 P 7af3795ebc594d0e8ad20fecd637b0cc [sys.catalog]: configured and running, proceeding with master startup.
I20260923 15:05:53.487295 2090 consensus_queue.cc:237] T 00000000000000000000000000000000 P 7af3795ebc594d0e8ad20fecd637b0cc [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7af3795ebc594d0e8ad20fecd637b0cc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40049 } }
I20260923 15:05:53.487687 2090 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7af3795ebc594d0e8ad20fecd637b0cc [sys.catalog]: SysCatalogTable state changed. Reason: New leader 7af3795ebc594d0e8ad20fecd637b0cc. Latest consensus state: current_term: 1 leader_uuid: "7af3795ebc594d0e8ad20fecd637b0cc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7af3795ebc594d0e8ad20fecd637b0cc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40049 } } }
I20260923 15:05:53.487712 2092 sys_catalog.cc:455] T 00000000000000000000000000000000 P 7af3795ebc594d0e8ad20fecd637b0cc [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "7af3795ebc594d0e8ad20fecd637b0cc" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "7af3795ebc594d0e8ad20fecd637b0cc" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40049 } } }
I20260923 15:05:53.487836 2090 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7af3795ebc594d0e8ad20fecd637b0cc [sys.catalog]: This master's current role is: LEADER
I20260923 15:05:53.487843 2092 sys_catalog.cc:458] T 00000000000000000000000000000000 P 7af3795ebc594d0e8ad20fecd637b0cc [sys.catalog]: This master's current role is: LEADER
I20260923 15:05:53.488085 2099 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 15:05:53.488615 2099 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 15:05:53.490423 2099 catalog_manager.cc:1347] Generated new cluster ID: 5117cf872cb64b569c83acbd600871b7
I20260923 15:05:53.491065 2099 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 23 15:05:53 dist-test-slave-cnb0 krb5kdc[2004](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175953, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260923 15:05:53.495669 2089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:53.484388 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.131:54053 (local address 127.0.0.1:40049)
0923 15:05:53.484593 (+ 205us) server_negotiation.cc:206] Beginning negotiation
0923 15:05:53.484602 (+ 9us) server_negotiation.cc:399] Waiting for connection header
0923 15:05:53.484639 (+ 37us) server_negotiation.cc:407] Connection header received
0923 15:05:53.485648 (+ 1009us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:05:53.485657 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:05:53.485726 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:05:53.485872 (+ 146us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:05:53.486620 (+ 748us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:05:53.488115 (+ 1495us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:05:53.489204 (+ 1089us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:05:53.489720 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:05:53.489836 (+ 116us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:05:53.492308 (+ 2472us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:05:53.492311 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:05:53.492312 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:05:53.492322 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:05:53.494000 (+ 1678us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:05:53.494409 (+ 409us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:05:53.494412 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:05:53.494412 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:05:53.494500 (+ 88us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:05:53.494732 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:05:53.494733 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:05:53.494734 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:05:53.495052 (+ 318us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:05:53.495255 (+ 203us) server_negotiation.cc:1035] Waiting for connection context
0923 15:05:53.495343 (+ 88us) server_negotiation.cc:299] Negotiation successful
0923 15:05:53.495447 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":155,"thread_start_us":91,"threads_started":1}
I20260923 15:05:53.496833 13614 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 15:05:53.505888 2089 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:53.498351 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.131:47543 (local address 127.0.0.1:40049)
0923 15:05:53.498392 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0923 15:05:53.498395 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:05:53.498545 (+ 150us) server_negotiation.cc:407] Connection header received
0923 15:05:53.498620 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:05:53.498621 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:05:53.498632 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:05:53.498694 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:05:53.499378 (+ 684us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:05:53.499929 (+ 551us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:05:53.500753 (+ 824us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:05:53.501005 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:05:53.501110 (+ 105us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:05:53.502499 (+ 1389us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:05:53.502500 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:05:53.502501 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:05:53.502509 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:05:53.504186 (+ 1677us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:05:53.504620 (+ 434us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:05:53.504621 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:05:53.504622 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:05:53.504688 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:05:53.504970 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:05:53.504971 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:05:53.504972 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:05:53.505104 (+ 132us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:05:53.505228 (+ 124us) server_negotiation.cc:1035] Waiting for connection context
0923 15:05:53.505678 (+ 450us) server_negotiation.cc:299] Negotiation successful
0923 15:05:53.505757 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260923 15:05:53.505756 2099 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 15:05:53.506654 2099 catalog_manager.cc:1504] Loading token signing keys...
I20260923 15:05:53.512732 2099 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 7af3795ebc594d0e8ad20fecd637b0cc: Generated new TSK 0
I20260923 15:05:53.512948 2099 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 15:05:53.515286 13614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKC4soy/build/release/bin/kudu with pid 2017
W20260923 15:05:53.519352 2002 connection.cc:535] client connection to 127.0.0.1:40049 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:40049): Connection reset by peer (error 104)
2026-09-23T15:05:53Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/6 (1328 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/7
Loading random data
Initializing database '/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790175691491353-13614-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 15:05:53 dist-test-slave-cnb0 krb5kdc[2116](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 15:05:53 dist-test-slave-cnb0 krb5kdc[2116](info): set up 2 sockets
Sep 23 15:05:53 dist-test-slave-cnb0 krb5kdc[2116](info): commencing operation
krb5kdc: starting...
W20260923 15:05:54.572521 13614 mini_kdc.cc:121] Time spent starting KDC: real 1.039s user 0.002s sys 0.003s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790175691491353-13614-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790175691491353-13614-0/krb5kdc/test-user.keytab.
Sep 23 15:05:54 dist-test-slave-cnb0 krb5kdc[2116](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175954, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T15:05:54Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T15:05:54Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790175691491353-13614-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790175691491353-13614-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260923 15:05:54.703243 13614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKC4soy/build/release/bin/kudu
/tmp/dist-test-taskKC4soy/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790175691491353-13614-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790175691491353-13614-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790175691491353-13614-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790175691491353-13614-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40757
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790175691491353-13614-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:44695
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:40757
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790175691491353-13614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790175691491353-13614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790175691491353-13614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:05:54.788583 2129 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:05:54.788866 2129 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:05:54.788909 2129 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:05:54.790267 2129 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 15:05:54.790355 2129 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:05:54.790380 2129 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:05:54.790410 2129 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 15:05:54.790442 2129 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 15:05:54.792137 2129 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:44695
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790175691491353-13614-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790175691491353-13614-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:40757
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790175691491353-13614-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:40757
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790175691491353-13614-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2129
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790175691491353-13614-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:05:54.792464 2129 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:05:54.792757 2129 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:05:54.798139 2135 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:05:54.798193 2137 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:05:54.798158 2134 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:05:54.798378 2129 server_base.cc:1034] running on GCE node
I20260923 15:05:54.798926 2129 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:05:54.799358 2129 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:05:54.800695 2129 hybrid_clock.cc:648] HybridClock initialized: now 1790175954800675 us; error 50 us; skew 500 ppm
Sep 23 15:05:54 dist-test-slave-cnb0 krb5kdc[2116](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175954, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:05:54.803867 2129 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260923 15:05:54.804468 2129 webserver.cc:460] Webserver started at http://127.0.0.1:35065/ using document root <none> and password file <none>
I20260923 15:05:54.804744 2129 fs_manager.cc:359] Metadata directory not provided
I20260923 15:05:54.804795 2129 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:05:54.804953 2129 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:05:54.806129 2129 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790175691491353-13614-0/minicluster-data/master-0/data/instance:
uuid: "e43c63082e4c4bf09dcc5a270d94006c"
format_stamp: "Formatted at 2026-09-23 15:05:54 on dist-test-slave-cnb0"
server_key: "b402cfa8d5c08fe5bc0242ac1000bba8"
server_key_iv: "8085c0d2fd6b617620199caf9d55cb99"
server_key_version: "encryptionkey@0"
I20260923 15:05:54.806516 2129 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790175691491353-13614-0/minicluster-data/master-0/wal/instance:
uuid: "e43c63082e4c4bf09dcc5a270d94006c"
format_stamp: "Formatted at 2026-09-23 15:05:54 on dist-test-slave-cnb0"
server_key: "b402cfa8d5c08fe5bc0242ac1000bba8"
server_key_iv: "8085c0d2fd6b617620199caf9d55cb99"
server_key_version: "encryptionkey@0"
I20260923 15:05:54.808090 2129 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260923 15:05:54.809048 2144 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:05:54.809325 2129 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260923 15:05:54.809427 2129 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790175691491353-13614-0/minicluster-data/master-0/data,/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790175691491353-13614-0/minicluster-data/master-0/wal
uuid: "e43c63082e4c4bf09dcc5a270d94006c"
format_stamp: "Formatted at 2026-09-23 15:05:54 on dist-test-slave-cnb0"
server_key: "b402cfa8d5c08fe5bc0242ac1000bba8"
server_key_iv: "8085c0d2fd6b617620199caf9d55cb99"
server_key_version: "encryptionkey@0"
I20260923 15:05:54.809494 2129 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790175691491353-13614-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790175691491353-13614-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790175691491353-13614-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:05:54.816883 2129 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:05:54.819563 2129 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:05:54.819690 2129 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:05:54.823830 2129 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:40757
I20260923 15:05:54.823895 2196 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:40757 every 8 connection(s)
I20260923 15:05:54.824280 2129 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790175691491353-13614-0/minicluster-data/master-0/data/info.pb
I20260923 15:05:54.825263 2197 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:05:54.827884 2197 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e43c63082e4c4bf09dcc5a270d94006c: Bootstrap starting.
I20260923 15:05:54.828534 2197 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e43c63082e4c4bf09dcc5a270d94006c: Neither blocks nor log segments found. Creating new log.
I20260923 15:05:54.828868 13614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKC4soy/build/release/bin/kudu as pid 2129
I20260923 15:05:54.828941 2197 log.cc:824] T 00000000000000000000000000000000 P e43c63082e4c4bf09dcc5a270d94006c: Log is configured to *not* fsync() on all Append() calls
I20260923 15:05:54.829007 13614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_7.1790175691491353-13614-0/minicluster-data/master-0/wal/instance
I20260923 15:05:54.829250 13614 external_mini_cluster.cc:1428] Setting key 9e28e582ffeaa5cf962868863a2a9182
I20260923 15:05:54.829869 2197 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e43c63082e4c4bf09dcc5a270d94006c: No bootstrap required, opened a new log
I20260923 15:05:54.832844 2197 raft_consensus.cc:348] T 00000000000000000000000000000000 P e43c63082e4c4bf09dcc5a270d94006c [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e43c63082e4c4bf09dcc5a270d94006c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40757 } }
I20260923 15:05:54.833035 2197 raft_consensus.cc:374] T 00000000000000000000000000000000 P e43c63082e4c4bf09dcc5a270d94006c [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:05:54.833066 2197 raft_consensus.cc:729] T 00000000000000000000000000000000 P e43c63082e4c4bf09dcc5a270d94006c [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e43c63082e4c4bf09dcc5a270d94006c, State: Initialized, Role: FOLLOWER
I20260923 15:05:54.833199 2197 consensus_queue.cc:260] T 00000000000000000000000000000000 P e43c63082e4c4bf09dcc5a270d94006c [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e43c63082e4c4bf09dcc5a270d94006c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40757 } }
I20260923 15:05:54.833289 2197 raft_consensus.cc:388] T 00000000000000000000000000000000 P e43c63082e4c4bf09dcc5a270d94006c [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 15:05:54.833334 2197 raft_consensus.cc:482] T 00000000000000000000000000000000 P e43c63082e4c4bf09dcc5a270d94006c [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 15:05:54.833381 2197 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e43c63082e4c4bf09dcc5a270d94006c [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:05:54.834053 2197 raft_consensus.cc:504] T 00000000000000000000000000000000 P e43c63082e4c4bf09dcc5a270d94006c [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e43c63082e4c4bf09dcc5a270d94006c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40757 } }
I20260923 15:05:54.834192 2197 leader_election.cc:302] T 00000000000000000000000000000000 P e43c63082e4c4bf09dcc5a270d94006c [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: e43c63082e4c4bf09dcc5a270d94006c; no voters:
I20260923 15:05:54.834373 2197 leader_election.cc:288] T 00000000000000000000000000000000 P e43c63082e4c4bf09dcc5a270d94006c [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 15:05:54.834460 2202 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e43c63082e4c4bf09dcc5a270d94006c [term 1 FOLLOWER]: Leader election won for term 1
I20260923 15:05:54.834654 2197 sys_catalog.cc:564] T 00000000000000000000000000000000 P e43c63082e4c4bf09dcc5a270d94006c [sys.catalog]: configured and running, proceeding with master startup.
I20260923 15:05:54.834707 2202 raft_consensus.cc:686] T 00000000000000000000000000000000 P e43c63082e4c4bf09dcc5a270d94006c [term 1 LEADER]: Becoming Leader. State: Replica: e43c63082e4c4bf09dcc5a270d94006c, State: Running, Role: LEADER
I20260923 15:05:54.834808 2202 consensus_queue.cc:237] T 00000000000000000000000000000000 P e43c63082e4c4bf09dcc5a270d94006c [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e43c63082e4c4bf09dcc5a270d94006c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40757 } }
I20260923 15:05:54.835206 2202 sys_catalog.cc:455] T 00000000000000000000000000000000 P e43c63082e4c4bf09dcc5a270d94006c [sys.catalog]: SysCatalogTable state changed. Reason: New leader e43c63082e4c4bf09dcc5a270d94006c. Latest consensus state: current_term: 1 leader_uuid: "e43c63082e4c4bf09dcc5a270d94006c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e43c63082e4c4bf09dcc5a270d94006c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40757 } } }
I20260923 15:05:54.835292 2202 sys_catalog.cc:458] T 00000000000000000000000000000000 P e43c63082e4c4bf09dcc5a270d94006c [sys.catalog]: This master's current role is: LEADER
I20260923 15:05:54.835513 2210 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 15:05:54.836099 2210 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 15:05:54.836177 2203 sys_catalog.cc:455] T 00000000000000000000000000000000 P e43c63082e4c4bf09dcc5a270d94006c [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "e43c63082e4c4bf09dcc5a270d94006c" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "e43c63082e4c4bf09dcc5a270d94006c" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 40757 } } }
I20260923 15:05:54.836298 2203 sys_catalog.cc:458] T 00000000000000000000000000000000 P e43c63082e4c4bf09dcc5a270d94006c [sys.catalog]: This master's current role is: LEADER
I20260923 15:05:54.836850 2201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:54.829853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.131:58981 (local address 127.0.0.1:40757)
0923 15:05:54.830068 (+ 215us) server_negotiation.cc:206] Beginning negotiation
0923 15:05:54.830078 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0923 15:05:54.830115 (+ 37us) server_negotiation.cc:407] Connection header received
0923 15:05:54.831271 (+ 1156us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:05:54.831281 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:05:54.831355 (+ 74us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:05:54.831521 (+ 166us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:05:54.831737 (+ 216us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:05:54.833189 (+ 1452us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:05:54.834146 (+ 957us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:05:54.834799 (+ 653us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:05:54.834962 (+ 163us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:05:54.835964 (+ 1002us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:05:54.835969 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:05:54.835969 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 15:05:54.836221 (+ 252us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:05:54.836268 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 15:05:54.836287 (+ 19us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:05:54.836379 (+ 92us) server_negotiation.cc:1035] Waiting for connection context
0923 15:05:54.836584 (+ 205us) server_negotiation.cc:299] Negotiation successful
0923 15:05:54.836619 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":88,"threads_started":1}
I20260923 15:05:54.837927 13614 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 15:05:54.840158 2210 catalog_manager.cc:1347] Generated new cluster ID: 5bfc9e2565f04e249b80a8137cf1066c
I20260923 15:05:54.840272 2210 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 15:05:54.843339 2201 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:54.839652 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.131:60921 (local address 127.0.0.1:40757)
0923 15:05:54.839697 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0923 15:05:54.839699 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:05:54.840030 (+ 331us) server_negotiation.cc:407] Connection header received
0923 15:05:54.840097 (+ 67us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:05:54.840098 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:05:54.840138 (+ 40us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:05:54.840206 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:05:54.840411 (+ 205us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:05:54.840905 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:05:54.842046 (+ 1141us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:05:54.842283 (+ 237us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:05:54.842349 (+ 66us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:05:54.843058 (+ 709us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:05:54.843059 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:05:54.843059 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 15:05:54.843065 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:05:54.843082 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 15:05:54.843087 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:05:54.843155 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0923 15:05:54.843243 (+ 88us) server_negotiation.cc:299] Negotiation successful
0923 15:05:54.843255 (+ 12us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":20,"server-negotiator.queue_time_us":22}
I20260923 15:05:54.859344 2210 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 15:05:54.860009 2210 catalog_manager.cc:1504] Loading token signing keys...
I20260923 15:05:54.869716 2210 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e43c63082e4c4bf09dcc5a270d94006c: Generated new TSK 0
I20260923 15:05:54.869982 2210 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 15:05:54.873708 13614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKC4soy/build/release/bin/kudu with pid 2129
W20260923 15:05:54.878768 2114 connection.cc:535] client connection to 127.0.0.1:40757 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:40757): Connection reset by peer (error 104)
2026-09-23T15:05:54Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/7 (1359 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/8
Loading random data
Initializing database '/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790175691491353-13614-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 15:05:54 dist-test-slave-cnb0 krb5kdc[2228](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 15:05:54 dist-test-slave-cnb0 krb5kdc[2228](info): set up 2 sockets
Sep 23 15:05:54 dist-test-slave-cnb0 krb5kdc[2228](info): commencing operation
krb5kdc: starting...
W20260923 15:05:55.976414 13614 mini_kdc.cc:121] Time spent starting KDC: real 1.083s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790175691491353-13614-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790175691491353-13614-0/krb5kdc/test-user.keytab.
Sep 23 15:05:56 dist-test-slave-cnb0 krb5kdc[2228](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175956, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T15:05:56Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T15:05:56Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790175691491353-13614-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790175691491353-13614-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260923 15:05:56.103204 13614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKC4soy/build/release/bin/kudu
/tmp/dist-test-taskKC4soy/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790175691491353-13614-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790175691491353-13614-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790175691491353-13614-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790175691491353-13614-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36251
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790175691491353-13614-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:36979
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:36251
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790175691491353-13614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790175691491353-13614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790175691491353-13614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:05:56.186687 2241 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:05:56.186959 2241 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:05:56.186986 2241 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:05:56.188272 2241 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 15:05:56.188318 2241 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:05:56.188333 2241 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:05:56.188345 2241 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 15:05:56.188357 2241 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 15:05:56.189868 2241 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:36979
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790175691491353-13614-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790175691491353-13614-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:36251
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790175691491353-13614-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:36251
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790175691491353-13614-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2241
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790175691491353-13614-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:05:56.190097 2241 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:05:56.190287 2241 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:05:56.195322 2246 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:05:56.195317 2247 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:05:56.195322 2249 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:05:56.195417 2241 server_base.cc:1034] running on GCE node
I20260923 15:05:56.195811 2241 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:05:56.196126 2241 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:05:56.197324 2241 hybrid_clock.cc:648] HybridClock initialized: now 1790175956197317 us; error 38 us; skew 500 ppm
Sep 23 15:05:56 dist-test-slave-cnb0 krb5kdc[2228](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175956, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:05:56.205476 2241 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260923 15:05:56.206019 2241 webserver.cc:460] Webserver started at http://127.0.0.1:38219/ using document root <none> and password file <none>
I20260923 15:05:56.206292 2241 fs_manager.cc:359] Metadata directory not provided
I20260923 15:05:56.206378 2241 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:05:56.206547 2241 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:05:56.207751 2241 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790175691491353-13614-0/minicluster-data/master-0/data/instance:
uuid: "32abf93a45bc4d88a31aff34b29f607d"
format_stamp: "Formatted at 2026-09-23 15:05:56 on dist-test-slave-cnb0"
server_key: "9d1c4203101b96ad1abcdfbc9a79f94f"
server_key_iv: "3f28facf85fb4d7f4a19cc1213d104f7"
server_key_version: "encryptionkey@0"
I20260923 15:05:56.208123 2241 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790175691491353-13614-0/minicluster-data/master-0/wal/instance:
uuid: "32abf93a45bc4d88a31aff34b29f607d"
format_stamp: "Formatted at 2026-09-23 15:05:56 on dist-test-slave-cnb0"
server_key: "9d1c4203101b96ad1abcdfbc9a79f94f"
server_key_iv: "3f28facf85fb4d7f4a19cc1213d104f7"
server_key_version: "encryptionkey@0"
I20260923 15:05:56.209543 2241 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260923 15:05:56.210440 2256 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:05:56.210703 2241 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260923 15:05:56.210855 2241 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790175691491353-13614-0/minicluster-data/master-0/data,/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790175691491353-13614-0/minicluster-data/master-0/wal
uuid: "32abf93a45bc4d88a31aff34b29f607d"
format_stamp: "Formatted at 2026-09-23 15:05:56 on dist-test-slave-cnb0"
server_key: "9d1c4203101b96ad1abcdfbc9a79f94f"
server_key_iv: "3f28facf85fb4d7f4a19cc1213d104f7"
server_key_version: "encryptionkey@0"
I20260923 15:05:56.210997 2241 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790175691491353-13614-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790175691491353-13614-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790175691491353-13614-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:05:56.214542 2241 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:05:56.217010 2241 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:05:56.217232 2241 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:05:56.221766 2241 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:36251
I20260923 15:05:56.221802 2309 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:36251 every 8 connection(s)
I20260923 15:05:56.222164 2241 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790175691491353-13614-0/minicluster-data/master-0/data/info.pb
I20260923 15:05:56.223250 2310 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:05:56.225608 2310 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 32abf93a45bc4d88a31aff34b29f607d: Bootstrap starting.
I20260923 15:05:56.226218 2310 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 32abf93a45bc4d88a31aff34b29f607d: Neither blocks nor log segments found. Creating new log.
I20260923 15:05:56.226521 2310 log.cc:824] T 00000000000000000000000000000000 P 32abf93a45bc4d88a31aff34b29f607d: Log is configured to *not* fsync() on all Append() calls
I20260923 15:05:56.227413 2310 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 32abf93a45bc4d88a31aff34b29f607d: No bootstrap required, opened a new log
I20260923 15:05:56.228528 13614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKC4soy/build/release/bin/kudu as pid 2241
I20260923 15:05:56.228708 13614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_8.1790175691491353-13614-0/minicluster-data/master-0/wal/instance
I20260923 15:05:56.228907 13614 external_mini_cluster.cc:1428] Setting key b73668293a31bc873096f596b053d365
I20260923 15:05:56.229547 2310 raft_consensus.cc:348] T 00000000000000000000000000000000 P 32abf93a45bc4d88a31aff34b29f607d [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32abf93a45bc4d88a31aff34b29f607d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36251 } }
I20260923 15:05:56.229772 2310 raft_consensus.cc:374] T 00000000000000000000000000000000 P 32abf93a45bc4d88a31aff34b29f607d [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:05:56.229830 2310 raft_consensus.cc:729] T 00000000000000000000000000000000 P 32abf93a45bc4d88a31aff34b29f607d [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 32abf93a45bc4d88a31aff34b29f607d, State: Initialized, Role: FOLLOWER
I20260923 15:05:56.229964 2310 consensus_queue.cc:260] T 00000000000000000000000000000000 P 32abf93a45bc4d88a31aff34b29f607d [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32abf93a45bc4d88a31aff34b29f607d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36251 } }
I20260923 15:05:56.230041 2310 raft_consensus.cc:388] T 00000000000000000000000000000000 P 32abf93a45bc4d88a31aff34b29f607d [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 15:05:56.230063 2310 raft_consensus.cc:482] T 00000000000000000000000000000000 P 32abf93a45bc4d88a31aff34b29f607d [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 15:05:56.230140 2310 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 32abf93a45bc4d88a31aff34b29f607d [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:05:56.231009 2310 raft_consensus.cc:504] T 00000000000000000000000000000000 P 32abf93a45bc4d88a31aff34b29f607d [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32abf93a45bc4d88a31aff34b29f607d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36251 } }
I20260923 15:05:56.231129 2310 leader_election.cc:302] T 00000000000000000000000000000000 P 32abf93a45bc4d88a31aff34b29f607d [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 32abf93a45bc4d88a31aff34b29f607d; no voters:
I20260923 15:05:56.231326 2310 leader_election.cc:288] T 00000000000000000000000000000000 P 32abf93a45bc4d88a31aff34b29f607d [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 15:05:56.231659 2310 sys_catalog.cc:564] T 00000000000000000000000000000000 P 32abf93a45bc4d88a31aff34b29f607d [sys.catalog]: configured and running, proceeding with master startup.
I20260923 15:05:56.231698 2314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:56.229607 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.131:51171 (local address 127.0.0.1:36251)
0923 15:05:56.229842 (+ 235us) server_negotiation.cc:206] Beginning negotiation
0923 15:05:56.229852 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0923 15:05:56.229890 (+ 38us) server_negotiation.cc:407] Connection header received
0923 15:05:56.230059 (+ 169us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:05:56.230067 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:05:56.230101 (+ 34us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:05:56.230294 (+ 193us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:05:56.231192 (+ 898us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:05:56.231196 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:05:56.231196 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 15:05:56.231201 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:05:56.231252 (+ 51us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 15:05:56.231267 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:05:56.231367 (+ 100us) server_negotiation.cc:1035] Waiting for connection context
0923 15:05:56.231456 (+ 89us) server_negotiation.cc:299] Negotiation successful
0923 15:05:56.231512 (+ 56us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":178,"thread_start_us":145,"threads_started":1}
I20260923 15:05:56.231345 2315 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 32abf93a45bc4d88a31aff34b29f607d [term 1 FOLLOWER]: Leader election won for term 1
I20260923 15:05:56.232034 2315 raft_consensus.cc:686] T 00000000000000000000000000000000 P 32abf93a45bc4d88a31aff34b29f607d [term 1 LEADER]: Becoming Leader. State: Replica: 32abf93a45bc4d88a31aff34b29f607d, State: Running, Role: LEADER
I20260923 15:05:56.232187 2315 consensus_queue.cc:237] T 00000000000000000000000000000000 P 32abf93a45bc4d88a31aff34b29f607d [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32abf93a45bc4d88a31aff34b29f607d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36251 } }
I20260923 15:05:56.232630 13614 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 15:05:56.232831 2316 sys_catalog.cc:455] T 00000000000000000000000000000000 P 32abf93a45bc4d88a31aff34b29f607d [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "32abf93a45bc4d88a31aff34b29f607d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32abf93a45bc4d88a31aff34b29f607d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36251 } } }
I20260923 15:05:56.232941 2316 sys_catalog.cc:458] T 00000000000000000000000000000000 P 32abf93a45bc4d88a31aff34b29f607d [sys.catalog]: This master's current role is: LEADER
I20260923 15:05:56.233070 2316 sys_catalog.cc:455] T 00000000000000000000000000000000 P 32abf93a45bc4d88a31aff34b29f607d [sys.catalog]: SysCatalogTable state changed. Reason: New leader 32abf93a45bc4d88a31aff34b29f607d. Latest consensus state: current_term: 1 leader_uuid: "32abf93a45bc4d88a31aff34b29f607d" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "32abf93a45bc4d88a31aff34b29f607d" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 36251 } } }
I20260923 15:05:56.233122 2316 sys_catalog.cc:458] T 00000000000000000000000000000000 P 32abf93a45bc4d88a31aff34b29f607d [sys.catalog]: This master's current role is: LEADER
I20260923 15:05:56.233217 2330 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 15:05:56.233804 2330 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 15:05:56.235759 2314 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:56.234314 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.131:36145 (local address 127.0.0.1:36251)
0923 15:05:56.234350 (+ 36us) server_negotiation.cc:206] Beginning negotiation
0923 15:05:56.234352 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:05:56.234660 (+ 308us) server_negotiation.cc:407] Connection header received
0923 15:05:56.234726 (+ 66us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:05:56.234727 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:05:56.234733 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:05:56.234784 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:05:56.235477 (+ 693us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:05:56.235478 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:05:56.235479 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 15:05:56.235483 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:05:56.235499 (+ 16us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 15:05:56.235504 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:05:56.235560 (+ 56us) server_negotiation.cc:1035] Waiting for connection context
0923 15:05:56.235672 (+ 112us) server_negotiation.cc:299] Negotiation successful
0923 15:05:56.235686 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260923 15:05:56.235788 2330 catalog_manager.cc:1347] Generated new cluster ID: f278ce2dbb7f4ceeabfd556649329ccd
I20260923 15:05:56.235826 2330 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 15:05:56.241290 2330 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 15:05:56.241482 2330 catalog_manager.cc:1504] Loading token signing keys...
I20260923 15:05:56.246325 2330 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 32abf93a45bc4d88a31aff34b29f607d: Generated new TSK 0
I20260923 15:05:56.246592 2330 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 15:05:56.255568 13614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKC4soy/build/release/bin/kudu with pid 2241
2026-09-23T15:05:56Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/8 (1381 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/9
Loading random data
Initializing database '/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790175691491353-13614-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 15:05:56 dist-test-slave-cnb0 krb5kdc[2341](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 15:05:56 dist-test-slave-cnb0 krb5kdc[2341](info): set up 2 sockets
Sep 23 15:05:56 dist-test-slave-cnb0 krb5kdc[2341](info): commencing operation
krb5kdc: starting...
W20260923 15:05:57.353063 13614 mini_kdc.cc:121] Time spent starting KDC: real 1.079s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790175691491353-13614-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790175691491353-13614-0/krb5kdc/test-user.keytab.
Sep 23 15:05:57 dist-test-slave-cnb0 krb5kdc[2341](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175957, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T15:05:57Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T15:05:57Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790175691491353-13614-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790175691491353-13614-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260923 15:05:57.483143 13614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKC4soy/build/release/bin/kudu
/tmp/dist-test-taskKC4soy/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790175691491353-13614-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790175691491353-13614-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790175691491353-13614-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790175691491353-13614-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43689
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790175691491353-13614-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:32905
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:43689
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790175691491353-13614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790175691491353-13614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790175691491353-13614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:05:57.563915 2354 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:05:57.564139 2354 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:05:57.564162 2354 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:05:57.565311 2354 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 15:05:57.565351 2354 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:05:57.565364 2354 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:05:57.565377 2354 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 15:05:57.565392 2354 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 15:05:57.566823 2354 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:32905
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790175691491353-13614-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790175691491353-13614-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:43689
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790175691491353-13614-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:43689
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790175691491353-13614-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2354
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790175691491353-13614-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:05:57.567040 2354 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:05:57.567214 2354 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:05:57.572129 2359 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:05:57.572114 2362 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:05:57.572142 2360 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:05:57.572399 2354 server_base.cc:1034] running on GCE node
I20260923 15:05:57.572677 2354 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:05:57.572921 2354 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:05:57.574101 2354 hybrid_clock.cc:648] HybridClock initialized: now 1790175957574092 us; error 35 us; skew 500 ppm
Sep 23 15:05:57 dist-test-slave-cnb0 krb5kdc[2341](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175957, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:05:57.576627 2354 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260923 15:05:57.577121 2354 webserver.cc:460] Webserver started at http://127.0.0.1:34859/ using document root <none> and password file <none>
I20260923 15:05:57.577307 2354 fs_manager.cc:359] Metadata directory not provided
I20260923 15:05:57.577347 2354 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:05:57.577438 2354 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:05:57.578461 2354 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790175691491353-13614-0/minicluster-data/master-0/data/instance:
uuid: "a0a673637cde4e629e6e448c969b1821"
format_stamp: "Formatted at 2026-09-23 15:05:57 on dist-test-slave-cnb0"
server_key: "0cbd4a524740238e8af427d7a02c8430"
server_key_iv: "3f92f4b3805c1a172cb77ca3331a8da1"
server_key_version: "encryptionkey@0"
I20260923 15:05:57.578768 2354 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790175691491353-13614-0/minicluster-data/master-0/wal/instance:
uuid: "a0a673637cde4e629e6e448c969b1821"
format_stamp: "Formatted at 2026-09-23 15:05:57 on dist-test-slave-cnb0"
server_key: "0cbd4a524740238e8af427d7a02c8430"
server_key_iv: "3f92f4b3805c1a172cb77ca3331a8da1"
server_key_version: "encryptionkey@0"
I20260923 15:05:57.580134 2354 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260923 15:05:57.580860 2369 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:05:57.581095 2354 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260923 15:05:57.581169 2354 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790175691491353-13614-0/minicluster-data/master-0/data,/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790175691491353-13614-0/minicluster-data/master-0/wal
uuid: "a0a673637cde4e629e6e448c969b1821"
format_stamp: "Formatted at 2026-09-23 15:05:57 on dist-test-slave-cnb0"
server_key: "0cbd4a524740238e8af427d7a02c8430"
server_key_iv: "3f92f4b3805c1a172cb77ca3331a8da1"
server_key_version: "encryptionkey@0"
I20260923 15:05:57.581264 2354 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790175691491353-13614-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790175691491353-13614-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790175691491353-13614-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:05:57.604445 2354 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:05:57.607041 2354 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:05:57.607205 2354 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:05:57.611186 2354 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:43689
I20260923 15:05:57.611248 2422 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:43689 every 8 connection(s)
I20260923 15:05:57.611624 2354 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790175691491353-13614-0/minicluster-data/master-0/data/info.pb
I20260923 15:05:57.612524 2423 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:05:57.614950 2423 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a0a673637cde4e629e6e448c969b1821: Bootstrap starting.
I20260923 15:05:57.615561 2423 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P a0a673637cde4e629e6e448c969b1821: Neither blocks nor log segments found. Creating new log.
I20260923 15:05:57.615859 2423 log.cc:824] T 00000000000000000000000000000000 P a0a673637cde4e629e6e448c969b1821: Log is configured to *not* fsync() on all Append() calls
I20260923 15:05:57.616636 2423 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P a0a673637cde4e629e6e448c969b1821: No bootstrap required, opened a new log
I20260923 15:05:57.618278 13614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKC4soy/build/release/bin/kudu as pid 2354
I20260923 15:05:57.618391 13614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_9.1790175691491353-13614-0/minicluster-data/master-0/wal/instance
I20260923 15:05:57.618559 13614 external_mini_cluster.cc:1428] Setting key 269760786d6a09a4a0de0dfd8a06ae1a
I20260923 15:05:57.618500 2423 raft_consensus.cc:348] T 00000000000000000000000000000000 P a0a673637cde4e629e6e448c969b1821 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0a673637cde4e629e6e448c969b1821" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43689 } }
I20260923 15:05:57.618670 2423 raft_consensus.cc:374] T 00000000000000000000000000000000 P a0a673637cde4e629e6e448c969b1821 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:05:57.618706 2423 raft_consensus.cc:729] T 00000000000000000000000000000000 P a0a673637cde4e629e6e448c969b1821 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: a0a673637cde4e629e6e448c969b1821, State: Initialized, Role: FOLLOWER
I20260923 15:05:57.618862 2423 consensus_queue.cc:260] T 00000000000000000000000000000000 P a0a673637cde4e629e6e448c969b1821 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0a673637cde4e629e6e448c969b1821" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43689 } }
I20260923 15:05:57.618999 2423 raft_consensus.cc:388] T 00000000000000000000000000000000 P a0a673637cde4e629e6e448c969b1821 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 15:05:57.619040 2423 raft_consensus.cc:482] T 00000000000000000000000000000000 P a0a673637cde4e629e6e448c969b1821 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 15:05:57.619089 2423 raft_consensus.cc:3037] T 00000000000000000000000000000000 P a0a673637cde4e629e6e448c969b1821 [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:05:57.619860 2423 raft_consensus.cc:504] T 00000000000000000000000000000000 P a0a673637cde4e629e6e448c969b1821 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0a673637cde4e629e6e448c969b1821" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43689 } }
I20260923 15:05:57.619971 2423 leader_election.cc:302] T 00000000000000000000000000000000 P a0a673637cde4e629e6e448c969b1821 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: a0a673637cde4e629e6e448c969b1821; no voters:
I20260923 15:05:57.620123 2423 leader_election.cc:288] T 00000000000000000000000000000000 P a0a673637cde4e629e6e448c969b1821 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 15:05:57.620167 2428 raft_consensus.cc:2781] T 00000000000000000000000000000000 P a0a673637cde4e629e6e448c969b1821 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 15:05:57.620326 2428 raft_consensus.cc:686] T 00000000000000000000000000000000 P a0a673637cde4e629e6e448c969b1821 [term 1 LEADER]: Becoming Leader. State: Replica: a0a673637cde4e629e6e448c969b1821, State: Running, Role: LEADER
I20260923 15:05:57.620407 2423 sys_catalog.cc:564] T 00000000000000000000000000000000 P a0a673637cde4e629e6e448c969b1821 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 15:05:57.620491 2428 consensus_queue.cc:237] T 00000000000000000000000000000000 P a0a673637cde4e629e6e448c969b1821 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0a673637cde4e629e6e448c969b1821" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43689 } }
I20260923 15:05:57.621042 2430 sys_catalog.cc:455] T 00000000000000000000000000000000 P a0a673637cde4e629e6e448c969b1821 [sys.catalog]: SysCatalogTable state changed. Reason: New leader a0a673637cde4e629e6e448c969b1821. Latest consensus state: current_term: 1 leader_uuid: "a0a673637cde4e629e6e448c969b1821" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0a673637cde4e629e6e448c969b1821" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43689 } } }
I20260923 15:05:57.621209 2430 sys_catalog.cc:458] T 00000000000000000000000000000000 P a0a673637cde4e629e6e448c969b1821 [sys.catalog]: This master's current role is: LEADER
I20260923 15:05:57.621567 2429 sys_catalog.cc:455] T 00000000000000000000000000000000 P a0a673637cde4e629e6e448c969b1821 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "a0a673637cde4e629e6e448c969b1821" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "a0a673637cde4e629e6e448c969b1821" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 43689 } } }
I20260923 15:05:57.621640 2429 sys_catalog.cc:458] T 00000000000000000000000000000000 P a0a673637cde4e629e6e448c969b1821 [sys.catalog]: This master's current role is: LEADER
I20260923 15:05:57.621865 2437 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 15:05:57.622462 2437 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 15:05:57.624131 2437 catalog_manager.cc:1347] Generated new cluster ID: f0c067f7f4f642bcbce2e4351678ad38
I20260923 15:05:57.624188 2437 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 23 15:05:57 dist-test-slave-cnb0 krb5kdc[2341](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175957, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.0.0.1@KRBTEST.COM
I20260923 15:05:57.631958 2427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:57.619082 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.131:34837 (local address 127.0.0.1:43689)
0923 15:05:57.619302 (+ 220us) server_negotiation.cc:206] Beginning negotiation
0923 15:05:57.619312 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0923 15:05:57.619374 (+ 62us) server_negotiation.cc:407] Connection header received
0923 15:05:57.619811 (+ 437us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:05:57.619821 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:05:57.619893 (+ 72us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:05:57.620049 (+ 156us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:05:57.621010 (+ 961us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:05:57.622657 (+ 1647us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:05:57.623808 (+ 1151us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:05:57.624460 (+ 652us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:05:57.624579 (+ 119us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:05:57.627635 (+ 3056us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:05:57.627640 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:05:57.627641 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:05:57.627657 (+ 16us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:05:57.629843 (+ 2186us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:05:57.630322 (+ 479us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:05:57.630325 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:05:57.630326 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:05:57.630458 (+ 132us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:05:57.630714 (+ 256us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:05:57.630715 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:05:57.630715 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:05:57.631131 (+ 416us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:05:57.631317 (+ 186us) server_negotiation.cc:1035] Waiting for connection context
0923 15:05:57.631602 (+ 285us) server_negotiation.cc:299] Negotiation successful
0923 15:05:57.631716 (+ 114us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":104,"threads_started":1}
I20260923 15:05:57.633329 2437 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 15:05:57.633383 13614 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 15:05:57.633864 2437 catalog_manager.cc:1504] Loading token signing keys...
I20260923 15:05:57.642822 2427 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:57.634948 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.131:55289 (local address 127.0.0.1:43689)
0923 15:05:57.634986 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0923 15:05:57.634988 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:05:57.635147 (+ 159us) server_negotiation.cc:407] Connection header received
0923 15:05:57.635209 (+ 62us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:05:57.635210 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:05:57.635221 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:05:57.635277 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:05:57.635899 (+ 622us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:05:57.636723 (+ 824us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:05:57.637778 (+ 1055us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:05:57.638134 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:05:57.638206 (+ 72us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:05:57.639741 (+ 1535us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:05:57.639743 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:05:57.639743 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:05:57.639751 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:05:57.641415 (+ 1664us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:05:57.641863 (+ 448us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:05:57.641864 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:05:57.641864 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:05:57.641925 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:05:57.642254 (+ 329us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:05:57.642255 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:05:57.642256 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:05:57.642387 (+ 131us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:05:57.642491 (+ 104us) server_negotiation.cc:1035] Waiting for connection context
0923 15:05:57.642651 (+ 160us) server_negotiation.cc:299] Negotiation successful
0923 15:05:57.642716 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260923 15:05:57.644413 2437 catalog_manager.cc:5868] T 00000000000000000000000000000000 P a0a673637cde4e629e6e448c969b1821: Generated new TSK 0
I20260923 15:05:57.644651 2437 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 15:05:57.647949 13614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKC4soy/build/release/bin/kudu with pid 2354
W20260923 15:05:57.652163 2339 connection.cc:535] client connection to 127.0.0.1:43689 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:43689): Connection reset by peer (error 104)
2026-09-23T15:05:57Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/9 (1392 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/10
Loading random data
Initializing database '/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790175691491353-13614-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 15:05:57 dist-test-slave-cnb0 krb5kdc[2454](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 15:05:57 dist-test-slave-cnb0 krb5kdc[2454](info): set up 2 sockets
Sep 23 15:05:57 dist-test-slave-cnb0 krb5kdc[2454](info): commencing operation
krb5kdc: starting...
W20260923 15:05:58.744148 13614 mini_kdc.cc:121] Time spent starting KDC: real 1.077s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790175691491353-13614-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790175691491353-13614-0/krb5kdc/test-user.keytab.
Sep 23 15:05:58 dist-test-slave-cnb0 krb5kdc[2454](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175958, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T15:05:58Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T15:05:58Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790175691491353-13614-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790175691491353-13614-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260923 15:05:58.878764 13614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKC4soy/build/release/bin/kudu
/tmp/dist-test-taskKC4soy/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790175691491353-13614-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790175691491353-13614-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790175691491353-13614-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790175691491353-13614-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45557
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790175691491353-13614-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:45065
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:45557
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790175691491353-13614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790175691491353-13614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790175691491353-13614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:05:58.962096 2467 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:05:58.962328 2467 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:05:58.962353 2467 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:05:58.963594 2467 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 15:05:58.963644 2467 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:05:58.963658 2467 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:05:58.963671 2467 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 15:05:58.963683 2467 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 15:05:58.965204 2467 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:45065
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790175691491353-13614-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790175691491353-13614-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:45557
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790175691491353-13614-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:45557
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790175691491353-13614-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2467
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790175691491353-13614-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:05:58.965422 2467 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:05:58.965611 2467 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:05:58.971094 2472 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:05:58.971086 2475 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:05:58.971086 2473 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:05:58.971664 2467 server_base.cc:1034] running on GCE node
I20260923 15:05:58.971946 2467 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:05:58.972319 2467 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:05:58.973483 2467 hybrid_clock.cc:648] HybridClock initialized: now 1790175958973467 us; error 32 us; skew 500 ppm
Sep 23 15:05:58 dist-test-slave-cnb0 krb5kdc[2454](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175958, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:05:58.976287 2467 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260923 15:05:58.976863 2467 webserver.cc:460] Webserver started at http://127.0.0.1:40065/ using document root <none> and password file <none>
I20260923 15:05:58.977123 2467 fs_manager.cc:359] Metadata directory not provided
I20260923 15:05:58.977195 2467 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:05:58.977349 2467 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:05:58.978477 2467 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790175691491353-13614-0/minicluster-data/master-0/data/instance:
uuid: "3cf05f7dbcda43edaff94501fc0af2fa"
format_stamp: "Formatted at 2026-09-23 15:05:58 on dist-test-slave-cnb0"
server_key: "816b63beb820f3f0bcbf62807de40db1"
server_key_iv: "b08ed9b29581087f4d8ca21fabc9ac64"
server_key_version: "encryptionkey@0"
I20260923 15:05:58.978848 2467 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790175691491353-13614-0/minicluster-data/master-0/wal/instance:
uuid: "3cf05f7dbcda43edaff94501fc0af2fa"
format_stamp: "Formatted at 2026-09-23 15:05:58 on dist-test-slave-cnb0"
server_key: "816b63beb820f3f0bcbf62807de40db1"
server_key_iv: "b08ed9b29581087f4d8ca21fabc9ac64"
server_key_version: "encryptionkey@0"
I20260923 15:05:58.980388 2467 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260923 15:05:58.981380 2482 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:05:58.981755 2467 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260923 15:05:58.981884 2467 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790175691491353-13614-0/minicluster-data/master-0/data,/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790175691491353-13614-0/minicluster-data/master-0/wal
uuid: "3cf05f7dbcda43edaff94501fc0af2fa"
format_stamp: "Formatted at 2026-09-23 15:05:58 on dist-test-slave-cnb0"
server_key: "816b63beb820f3f0bcbf62807de40db1"
server_key_iv: "b08ed9b29581087f4d8ca21fabc9ac64"
server_key_version: "encryptionkey@0"
I20260923 15:05:58.981973 2467 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790175691491353-13614-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790175691491353-13614-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790175691491353-13614-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:05:58.994165 2467 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:05:58.996870 2467 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:05:58.997081 2467 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:05:59.002126 2467 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:45557
I20260923 15:05:59.002215 2536 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:45557 every 8 connection(s)
I20260923 15:05:59.002620 2467 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790175691491353-13614-0/minicluster-data/master-0/data/info.pb
I20260923 15:05:59.003683 2537 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:05:59.004341 13614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKC4soy/build/release/bin/kudu as pid 2467
I20260923 15:05:59.004492 13614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_10.1790175691491353-13614-0/minicluster-data/master-0/wal/instance
I20260923 15:05:59.004727 13614 external_mini_cluster.cc:1428] Setting key ab414994920ad9da969548aa57ce279b
I20260923 15:05:59.007100 2537 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3cf05f7dbcda43edaff94501fc0af2fa: Bootstrap starting.
I20260923 15:05:59.007867 2537 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 3cf05f7dbcda43edaff94501fc0af2fa: Neither blocks nor log segments found. Creating new log.
I20260923 15:05:59.008224 2537 log.cc:824] T 00000000000000000000000000000000 P 3cf05f7dbcda43edaff94501fc0af2fa: Log is configured to *not* fsync() on all Append() calls
I20260923 15:05:59.009073 2537 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 3cf05f7dbcda43edaff94501fc0af2fa: No bootstrap required, opened a new log
I20260923 15:05:59.011322 2540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:59.005338 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.131:35945 (local address 127.0.0.1:45557)
0923 15:05:59.005525 (+ 187us) server_negotiation.cc:206] Beginning negotiation
0923 15:05:59.005533 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0923 15:05:59.005560 (+ 27us) server_negotiation.cc:407] Connection header received
0923 15:05:59.006008 (+ 448us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:05:59.006015 (+ 7us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:05:59.006069 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:05:59.006194 (+ 125us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:05:59.006469 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:05:59.007526 (+ 1057us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:05:59.008658 (+ 1132us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:05:59.009193 (+ 535us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:05:59.009302 (+ 109us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:05:59.010189 (+ 887us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:05:59.010194 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:05:59.010194 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 15:05:59.010381 (+ 187us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:05:59.010417 (+ 36us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 15:05:59.010429 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:05:59.010549 (+ 120us) server_negotiation.cc:1035] Waiting for connection context
0923 15:05:59.010661 (+ 112us) server_negotiation.cc:299] Negotiation successful
0923 15:05:59.010688 (+ 27us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":119,"thread_start_us":84,"threads_started":1}
I20260923 15:05:59.011498 2537 raft_consensus.cc:348] T 00000000000000000000000000000000 P 3cf05f7dbcda43edaff94501fc0af2fa [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3cf05f7dbcda43edaff94501fc0af2fa" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45557 } }
I20260923 15:05:59.011708 2537 raft_consensus.cc:374] T 00000000000000000000000000000000 P 3cf05f7dbcda43edaff94501fc0af2fa [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:05:59.011749 2537 raft_consensus.cc:729] T 00000000000000000000000000000000 P 3cf05f7dbcda43edaff94501fc0af2fa [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 3cf05f7dbcda43edaff94501fc0af2fa, State: Initialized, Role: FOLLOWER
I20260923 15:05:59.011878 2537 consensus_queue.cc:260] T 00000000000000000000000000000000 P 3cf05f7dbcda43edaff94501fc0af2fa [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3cf05f7dbcda43edaff94501fc0af2fa" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45557 } }
I20260923 15:05:59.011977 2537 raft_consensus.cc:388] T 00000000000000000000000000000000 P 3cf05f7dbcda43edaff94501fc0af2fa [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 15:05:59.012019 2537 raft_consensus.cc:482] T 00000000000000000000000000000000 P 3cf05f7dbcda43edaff94501fc0af2fa [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 15:05:59.012068 2537 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 3cf05f7dbcda43edaff94501fc0af2fa [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:05:59.012455 13614 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 15:05:59.012909 2537 raft_consensus.cc:504] T 00000000000000000000000000000000 P 3cf05f7dbcda43edaff94501fc0af2fa [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3cf05f7dbcda43edaff94501fc0af2fa" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45557 } }
I20260923 15:05:59.013036 2537 leader_election.cc:302] T 00000000000000000000000000000000 P 3cf05f7dbcda43edaff94501fc0af2fa [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 3cf05f7dbcda43edaff94501fc0af2fa; no voters:
I20260923 15:05:59.013222 2537 leader_election.cc:288] T 00000000000000000000000000000000 P 3cf05f7dbcda43edaff94501fc0af2fa [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 15:05:59.013334 2542 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 3cf05f7dbcda43edaff94501fc0af2fa [term 1 FOLLOWER]: Leader election won for term 1
I20260923 15:05:59.013496 2537 sys_catalog.cc:564] T 00000000000000000000000000000000 P 3cf05f7dbcda43edaff94501fc0af2fa [sys.catalog]: configured and running, proceeding with master startup.
I20260923 15:05:59.013499 2542 raft_consensus.cc:686] T 00000000000000000000000000000000 P 3cf05f7dbcda43edaff94501fc0af2fa [term 1 LEADER]: Becoming Leader. State: Replica: 3cf05f7dbcda43edaff94501fc0af2fa, State: Running, Role: LEADER
I20260923 15:05:59.013717 2542 consensus_queue.cc:237] T 00000000000000000000000000000000 P 3cf05f7dbcda43edaff94501fc0af2fa [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3cf05f7dbcda43edaff94501fc0af2fa" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45557 } }
I20260923 15:05:59.014015 2542 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3cf05f7dbcda43edaff94501fc0af2fa [sys.catalog]: SysCatalogTable state changed. Reason: New leader 3cf05f7dbcda43edaff94501fc0af2fa. Latest consensus state: current_term: 1 leader_uuid: "3cf05f7dbcda43edaff94501fc0af2fa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3cf05f7dbcda43edaff94501fc0af2fa" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45557 } } }
I20260923 15:05:59.014061 2542 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3cf05f7dbcda43edaff94501fc0af2fa [sys.catalog]: This master's current role is: LEADER
I20260923 15:05:59.014050 2544 sys_catalog.cc:455] T 00000000000000000000000000000000 P 3cf05f7dbcda43edaff94501fc0af2fa [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "3cf05f7dbcda43edaff94501fc0af2fa" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "3cf05f7dbcda43edaff94501fc0af2fa" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 45557 } } }
I20260923 15:05:59.014204 2544 sys_catalog.cc:458] T 00000000000000000000000000000000 P 3cf05f7dbcda43edaff94501fc0af2fa [sys.catalog]: This master's current role is: LEADER
I20260923 15:05:59.014400 2550 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 15:05:59.015492 2550 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 15:05:59.017418 2550 catalog_manager.cc:1347] Generated new cluster ID: 1880b1d9ed544f4b94bcb5ca85cbef05
I20260923 15:05:59.017473 2550 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 15:05:59.018814 2540 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:05:59.014978 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.131:46893 (local address 127.0.0.1:45557)
0923 15:05:59.015022 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0923 15:05:59.015024 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:05:59.015244 (+ 220us) server_negotiation.cc:407] Connection header received
0923 15:05:59.015342 (+ 98us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:05:59.015343 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:05:59.015354 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:05:59.015387 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:05:59.015662 (+ 275us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:05:59.016325 (+ 663us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:05:59.017299 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:05:59.017587 (+ 288us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:05:59.017671 (+ 84us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:05:59.018522 (+ 851us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:05:59.018524 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:05:59.018524 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 15:05:59.018529 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:05:59.018543 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 15:05:59.018547 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:05:59.018592 (+ 45us) server_negotiation.cc:1035] Waiting for connection context
0923 15:05:59.018731 (+ 139us) server_negotiation.cc:299] Negotiation successful
0923 15:05:59.018740 (+ 9us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260923 15:05:59.024488 2550 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 15:05:59.025240 2550 catalog_manager.cc:1504] Loading token signing keys...
I20260923 15:05:59.033318 2550 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 3cf05f7dbcda43edaff94501fc0af2fa: Generated new TSK 0
I20260923 15:05:59.033573 2550 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 15:05:59.041358 13614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKC4soy/build/release/bin/kudu with pid 2467
W20260923 15:05:59.046005 2452 connection.cc:535] client connection to 127.0.0.1:45557 recv error: Network error: failed to read from TLS socket (remote: 127.0.0.1:45557): Connection reset by peer (error 104)
2026-09-23T15:05:59Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/10 (1392 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/11
Loading random data
Initializing database '/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790175691491353-13614-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 15:05:59 dist-test-slave-cnb0 krb5kdc[2568](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 15:05:59 dist-test-slave-cnb0 krb5kdc[2568](info): set up 2 sockets
Sep 23 15:05:59 dist-test-slave-cnb0 krb5kdc[2568](info): commencing operation
krb5kdc: starting...
W20260923 15:06:00.100418 13614 mini_kdc.cc:121] Time spent starting KDC: real 1.040s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790175691491353-13614-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790175691491353-13614-0/krb5kdc/test-user.keytab.
Sep 23 15:06:00 dist-test-slave-cnb0 krb5kdc[2568](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175960, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T15:06:00Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T15:06:00Z Disabled control of system clock
WARNING: no policy specified for kudu/127.0.0.1@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.0.0.1@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790175691491353-13614-0/krb5kdc/kudu_127.0.0.1.keytab.
Entry for principal kudu/127.0.0.1 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790175691491353-13614-0/krb5kdc/kudu_127.0.0.1.keytab.
I20260923 15:06:00.231212 13614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKC4soy/build/release/bin/kudu
/tmp/dist-test-taskKC4soy/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790175691491353-13614-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790175691491353-13614-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790175691491353-13614-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790175691491353-13614-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39017
--webserver_interface=127.0.0.1
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790175691491353-13614-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.0.0.1:35551
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.0.0.1:39017
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790175691491353-13614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790175691491353-13614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790175691491353-13614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:06:00.317224 2581 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:06:00.317448 2581 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:06:00.317473 2581 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:06:00.318645 2581 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 15:06:00.318691 2581 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:06:00.318704 2581 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:06:00.318717 2581 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 15:06:00.318732 2581 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 15:06:00.320329 2581 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.0.0.1:35551
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790175691491353-13614-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790175691491353-13614-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.0.0.1:39017
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790175691491353-13614-0/krb5kdc/kudu_127.0.0.1.keytab
--principal=kudu/127.0.0.1
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.0.0.1:39017
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790175691491353-13614-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.0.0.1
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2581
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790175691491353-13614-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:06:00.320559 2581 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:06:00.320751 2581 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:06:00.326076 2589 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:06:00.326172 2586 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:06:00.326191 2581 server_base.cc:1034] running on GCE node
W20260923 15:06:00.326076 2587 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:06:00.326697 2581 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:06:00.327104 2581 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:06:00.328289 2581 hybrid_clock.cc:648] HybridClock initialized: now 1790175960328264 us; error 43 us; skew 500 ppm
Sep 23 15:06:00 dist-test-slave-cnb0 krb5kdc[2568](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175960, etypes {rep=17 tkt=17 ses=17}, kudu/127.0.0.1@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:06:00.331120 2581 init.cc:377] Logged in from keytab as kudu/127.0.0.1@KRBTEST.COM (short username kudu)
I20260923 15:06:00.331555 2581 webserver.cc:460] Webserver started at http://127.0.0.1:45441/ using document root <none> and password file <none>
I20260923 15:06:00.331774 2581 fs_manager.cc:359] Metadata directory not provided
I20260923 15:06:00.331817 2581 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:06:00.331910 2581 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:06:00.332965 2581 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790175691491353-13614-0/minicluster-data/master-0/data/instance:
uuid: "39c9ef0b9f3e4b0a9afcd836a82c78f3"
format_stamp: "Formatted at 2026-09-23 15:06:00 on dist-test-slave-cnb0"
server_key: "89466490f7b90369d13cb528fbaa26a7"
server_key_iv: "0312c82075d04369526dbaf37422f3b8"
server_key_version: "encryptionkey@0"
I20260923 15:06:00.333271 2581 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790175691491353-13614-0/minicluster-data/master-0/wal/instance:
uuid: "39c9ef0b9f3e4b0a9afcd836a82c78f3"
format_stamp: "Formatted at 2026-09-23 15:06:00 on dist-test-slave-cnb0"
server_key: "89466490f7b90369d13cb528fbaa26a7"
server_key_iv: "0312c82075d04369526dbaf37422f3b8"
server_key_version: "encryptionkey@0"
I20260923 15:06:00.334625 2581 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260923 15:06:00.335527 2596 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:06:00.335723 2581 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.000s
I20260923 15:06:00.335847 2581 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790175691491353-13614-0/minicluster-data/master-0/data,/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790175691491353-13614-0/minicluster-data/master-0/wal
uuid: "39c9ef0b9f3e4b0a9afcd836a82c78f3"
format_stamp: "Formatted at 2026-09-23 15:06:00 on dist-test-slave-cnb0"
server_key: "89466490f7b90369d13cb528fbaa26a7"
server_key_iv: "0312c82075d04369526dbaf37422f3b8"
server_key_version: "encryptionkey@0"
I20260923 15:06:00.335932 2581 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790175691491353-13614-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790175691491353-13614-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790175691491353-13614-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:06:00.339404 2581 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:06:00.341809 2581 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:06:00.341948 2581 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:06:00.346134 2581 rpc_server.cc:307] RPC server started. Bound to: 127.0.0.1:39017
I20260923 15:06:00.346168 2649 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.0.0.1:39017 every 8 connection(s)
I20260923 15:06:00.346514 2581 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790175691491353-13614-0/minicluster-data/master-0/data/info.pb
I20260923 15:06:00.347467 2650 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:06:00.349789 2650 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 39c9ef0b9f3e4b0a9afcd836a82c78f3: Bootstrap starting.
I20260923 15:06:00.350358 2650 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 39c9ef0b9f3e4b0a9afcd836a82c78f3: Neither blocks nor log segments found. Creating new log.
I20260923 15:06:00.350605 2650 log.cc:824] T 00000000000000000000000000000000 P 39c9ef0b9f3e4b0a9afcd836a82c78f3: Log is configured to *not* fsync() on all Append() calls
I20260923 15:06:00.351351 2650 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 39c9ef0b9f3e4b0a9afcd836a82c78f3: No bootstrap required, opened a new log
I20260923 15:06:00.352943 2650 raft_consensus.cc:348] T 00000000000000000000000000000000 P 39c9ef0b9f3e4b0a9afcd836a82c78f3 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39c9ef0b9f3e4b0a9afcd836a82c78f3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39017 } }
I20260923 15:06:00.353091 2650 raft_consensus.cc:374] T 00000000000000000000000000000000 P 39c9ef0b9f3e4b0a9afcd836a82c78f3 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:06:00.353114 2650 raft_consensus.cc:729] T 00000000000000000000000000000000 P 39c9ef0b9f3e4b0a9afcd836a82c78f3 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 39c9ef0b9f3e4b0a9afcd836a82c78f3, State: Initialized, Role: FOLLOWER
I20260923 15:06:00.353260 2650 consensus_queue.cc:260] T 00000000000000000000000000000000 P 39c9ef0b9f3e4b0a9afcd836a82c78f3 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39c9ef0b9f3e4b0a9afcd836a82c78f3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39017 } }
I20260923 15:06:00.353344 2650 raft_consensus.cc:388] T 00000000000000000000000000000000 P 39c9ef0b9f3e4b0a9afcd836a82c78f3 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 15:06:00.353367 2650 raft_consensus.cc:482] T 00000000000000000000000000000000 P 39c9ef0b9f3e4b0a9afcd836a82c78f3 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 15:06:00.353396 2650 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 39c9ef0b9f3e4b0a9afcd836a82c78f3 [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:06:00.353922 2650 raft_consensus.cc:504] T 00000000000000000000000000000000 P 39c9ef0b9f3e4b0a9afcd836a82c78f3 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39c9ef0b9f3e4b0a9afcd836a82c78f3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39017 } }
I20260923 15:06:00.354010 2650 leader_election.cc:302] T 00000000000000000000000000000000 P 39c9ef0b9f3e4b0a9afcd836a82c78f3 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 39c9ef0b9f3e4b0a9afcd836a82c78f3; no voters:
I20260923 15:06:00.354156 2650 leader_election.cc:288] T 00000000000000000000000000000000 P 39c9ef0b9f3e4b0a9afcd836a82c78f3 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 15:06:00.354251 2653 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 39c9ef0b9f3e4b0a9afcd836a82c78f3 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 15:06:00.354409 2650 sys_catalog.cc:564] T 00000000000000000000000000000000 P 39c9ef0b9f3e4b0a9afcd836a82c78f3 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 15:06:00.354437 2653 raft_consensus.cc:686] T 00000000000000000000000000000000 P 39c9ef0b9f3e4b0a9afcd836a82c78f3 [term 1 LEADER]: Becoming Leader. State: Replica: 39c9ef0b9f3e4b0a9afcd836a82c78f3, State: Running, Role: LEADER
I20260923 15:06:00.354552 2653 consensus_queue.cc:237] T 00000000000000000000000000000000 P 39c9ef0b9f3e4b0a9afcd836a82c78f3 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39c9ef0b9f3e4b0a9afcd836a82c78f3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39017 } }
I20260923 15:06:00.355026 2654 sys_catalog.cc:455] T 00000000000000000000000000000000 P 39c9ef0b9f3e4b0a9afcd836a82c78f3 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "39c9ef0b9f3e4b0a9afcd836a82c78f3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39c9ef0b9f3e4b0a9afcd836a82c78f3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39017 } } }
I20260923 15:06:00.355111 2654 sys_catalog.cc:458] T 00000000000000000000000000000000 P 39c9ef0b9f3e4b0a9afcd836a82c78f3 [sys.catalog]: This master's current role is: LEADER
I20260923 15:06:00.355165 2655 sys_catalog.cc:455] T 00000000000000000000000000000000 P 39c9ef0b9f3e4b0a9afcd836a82c78f3 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 39c9ef0b9f3e4b0a9afcd836a82c78f3. Latest consensus state: current_term: 1 leader_uuid: "39c9ef0b9f3e4b0a9afcd836a82c78f3" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "39c9ef0b9f3e4b0a9afcd836a82c78f3" member_type: VOTER last_known_addr { host: "127.0.0.1" port: 39017 } } }
I20260923 15:06:00.355237 2655 sys_catalog.cc:458] T 00000000000000000000000000000000 P 39c9ef0b9f3e4b0a9afcd836a82c78f3 [sys.catalog]: This master's current role is: LEADER
I20260923 15:06:00.355710 2662 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 15:06:00.356295 2662 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 15:06:00.356652 13614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKC4soy/build/release/bin/kudu as pid 2581
I20260923 15:06:00.356773 13614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_11.1790175691491353-13614-0/minicluster-data/master-0/wal/instance
I20260923 15:06:00.356966 13614 external_mini_cluster.cc:1428] Setting key a36c4ebadd932943fb169f02d1800c8d
I20260923 15:06:00.358626 2662 catalog_manager.cc:1347] Generated new cluster ID: 025471f9dcc04953a7c2dacf4f8e7e05
I20260923 15:06:00.358693 2662 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 15:06:00.359332 2672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:00.357469 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.131:36703 (local address 127.0.0.1:39017)
0923 15:06:00.357642 (+ 173us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:00.357652 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:00.357693 (+ 41us) server_negotiation.cc:407] Connection header received
0923 15:06:00.357800 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:00.357806 (+ 6us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:00.357841 (+ 35us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:00.357953 (+ 112us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:00.358732 (+ 779us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:06:00.358735 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:06:00.358736 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 15:06:00.358740 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:06:00.358788 (+ 48us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 15:06:00.358803 (+ 15us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:06:00.358841 (+ 38us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:00.359102 (+ 261us) server_negotiation.cc:299] Negotiation successful
0923 15:06:00.359150 (+ 48us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":113,"thread_start_us":86,"threads_started":1}
I20260923 15:06:00.360416 13614 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 15:06:00.363305 2672 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:00.362127 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 10.240.0.131:40483 (local address 127.0.0.1:39017)
0923 15:06:00.362169 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:00.362171 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:00.362180 (+ 9us) server_negotiation.cc:407] Connection header received
0923 15:06:00.362285 (+ 105us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:00.362286 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:00.362292 (+ 6us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:00.362354 (+ 62us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:00.363036 (+ 682us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:06:00.363038 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:06:00.363038 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 15:06:00.363042 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:06:00.363059 (+ 17us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 15:06:00.363064 (+ 5us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:06:00.363135 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:00.363220 (+ 85us) server_negotiation.cc:299] Negotiation successful
0923 15:06:00.363234 (+ 14us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260923 15:06:00.364130 2662 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 15:06:00.364276 2662 catalog_manager.cc:1504] Loading token signing keys...
I20260923 15:06:00.370517 2662 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 39c9ef0b9f3e4b0a9afcd836a82c78f3: Generated new TSK 0
I20260923 15:06:00.370818 2662 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 15:06:00.373229 13614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKC4soy/build/release/bin/kudu with pid 2581
2026-09-23T15:06:00Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/11 (1332 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/12
Loading random data
Initializing database '/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790175691491353-13614-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 15:06:00 dist-test-slave-cnb0 krb5kdc[2681](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 15:06:00 dist-test-slave-cnb0 krb5kdc[2681](info): set up 2 sockets
Sep 23 15:06:00 dist-test-slave-cnb0 krb5kdc[2681](info): commencing operation
krb5kdc: starting...
W20260923 15:06:01.432299 13614 mini_kdc.cc:121] Time spent starting KDC: real 1.040s user 0.000s sys 0.006s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790175691491353-13614-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790175691491353-13614-0/krb5kdc/test-user.keytab.
Sep 23 15:06:01 dist-test-slave-cnb0 krb5kdc[2681](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175961, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T15:06:01Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T15:06:01Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.75.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.75.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.75.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.190.keytab.
Entry for principal kudu/127.13.75.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.190.keytab.
I20260923 15:06:01.559592 13614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKC4soy/build/release/bin/kudu
/tmp/dist-test-taskKC4soy/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790175691491353-13614-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790175691491353-13614-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790175691491353-13614-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790175691491353-13614-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.75.190:41279
--webserver_interface=127.13.75.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.190.keytab
--principal=kudu/127.13.75.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.75.148:37843
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.75.190:41279
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790175691491353-13614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790175691491353-13614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790175691491353-13614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:06:01.643483 2694 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:06:01.643711 2694 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:06:01.643736 2694 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:06:01.644904 2694 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 15:06:01.644942 2694 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:06:01.644958 2694 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:06:01.644969 2694 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 15:06:01.644982 2694 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 15:06:01.646461 2694 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.75.148:37843
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790175691491353-13614-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790175691491353-13614-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.75.190:41279
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.190.keytab
--principal=kudu/127.13.75.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.75.190:41279
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790175691491353-13614-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.75.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2694
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790175691491353-13614-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:06:01.646677 2694 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:06:01.646860 2694 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:06:01.651950 2703 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:06:01.652098 2694 server_base.cc:1034] running on GCE node
W20260923 15:06:01.651928 2699 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:06:01.651906 2700 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:06:01.652460 2694 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:06:01.652812 2694 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:06:01.653981 2694 hybrid_clock.cc:648] HybridClock initialized: now 1790175961653965 us; error 32 us; skew 500 ppm
Sep 23 15:06:01 dist-test-slave-cnb0 krb5kdc[2681](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175961, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:06:01.662117 2694 init.cc:377] Logged in from keytab as kudu/127.13.75.190@KRBTEST.COM (short username kudu)
I20260923 15:06:01.662740 2694 webserver.cc:460] Webserver started at http://127.13.75.190:34925/ using document root <none> and password file <none>
I20260923 15:06:01.663049 2694 fs_manager.cc:359] Metadata directory not provided
I20260923 15:06:01.663138 2694 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:06:01.663316 2694 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:06:01.664817 2694 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790175691491353-13614-0/minicluster-data/master-0/data/instance:
uuid: "75a5aa3474fc4f85ad0ce399e61923df"
format_stamp: "Formatted at 2026-09-23 15:06:01 on dist-test-slave-cnb0"
server_key: "e839358cb4df6df07790d5bc04de63bd"
server_key_iv: "df9bed165a2e747d96a5f149ad7d42d8"
server_key_version: "encryptionkey@0"
I20260923 15:06:01.665369 2694 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790175691491353-13614-0/minicluster-data/master-0/wal/instance:
uuid: "75a5aa3474fc4f85ad0ce399e61923df"
format_stamp: "Formatted at 2026-09-23 15:06:01 on dist-test-slave-cnb0"
server_key: "e839358cb4df6df07790d5bc04de63bd"
server_key_iv: "df9bed165a2e747d96a5f149ad7d42d8"
server_key_version: "encryptionkey@0"
I20260923 15:06:01.667703 2694 fs_manager.cc:691] Time spent creating directory manager: real 0.002s user 0.004s sys 0.000s
I20260923 15:06:01.668898 2710 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:06:01.669441 2694 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:06:01.669566 2694 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790175691491353-13614-0/minicluster-data/master-0/data,/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790175691491353-13614-0/minicluster-data/master-0/wal
uuid: "75a5aa3474fc4f85ad0ce399e61923df"
format_stamp: "Formatted at 2026-09-23 15:06:01 on dist-test-slave-cnb0"
server_key: "e839358cb4df6df07790d5bc04de63bd"
server_key_iv: "df9bed165a2e747d96a5f149ad7d42d8"
server_key_version: "encryptionkey@0"
I20260923 15:06:01.669636 2694 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790175691491353-13614-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790175691491353-13614-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790175691491353-13614-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:06:01.700402 2694 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:06:01.702674 2694 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:06:01.702795 2694 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:06:01.707345 2694 rpc_server.cc:307] RPC server started. Bound to: 127.13.75.190:41279
I20260923 15:06:01.707422 2762 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.75.190:41279 every 8 connection(s)
I20260923 15:06:01.707774 2694 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790175691491353-13614-0/minicluster-data/master-0/data/info.pb
I20260923 15:06:01.708734 2763 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:06:01.711323 2763 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 75a5aa3474fc4f85ad0ce399e61923df: Bootstrap starting.
I20260923 15:06:01.711928 2763 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 75a5aa3474fc4f85ad0ce399e61923df: Neither blocks nor log segments found. Creating new log.
I20260923 15:06:01.712226 2763 log.cc:824] T 00000000000000000000000000000000 P 75a5aa3474fc4f85ad0ce399e61923df: Log is configured to *not* fsync() on all Append() calls
I20260923 15:06:01.713023 2763 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 75a5aa3474fc4f85ad0ce399e61923df: No bootstrap required, opened a new log
I20260923 15:06:01.714736 2763 raft_consensus.cc:348] T 00000000000000000000000000000000 P 75a5aa3474fc4f85ad0ce399e61923df [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75a5aa3474fc4f85ad0ce399e61923df" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 41279 } }
I20260923 15:06:01.714954 2763 raft_consensus.cc:374] T 00000000000000000000000000000000 P 75a5aa3474fc4f85ad0ce399e61923df [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:06:01.715011 2763 raft_consensus.cc:729] T 00000000000000000000000000000000 P 75a5aa3474fc4f85ad0ce399e61923df [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 75a5aa3474fc4f85ad0ce399e61923df, State: Initialized, Role: FOLLOWER
I20260923 15:06:01.715191 2763 consensus_queue.cc:260] T 00000000000000000000000000000000 P 75a5aa3474fc4f85ad0ce399e61923df [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75a5aa3474fc4f85ad0ce399e61923df" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 41279 } }
I20260923 15:06:01.715260 2763 raft_consensus.cc:388] T 00000000000000000000000000000000 P 75a5aa3474fc4f85ad0ce399e61923df [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 15:06:01.715319 2763 raft_consensus.cc:482] T 00000000000000000000000000000000 P 75a5aa3474fc4f85ad0ce399e61923df [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 15:06:01.715375 2763 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 75a5aa3474fc4f85ad0ce399e61923df [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:06:01.715538 13614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKC4soy/build/release/bin/kudu as pid 2694
I20260923 15:06:01.715647 13614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_12.1790175691491353-13614-0/minicluster-data/master-0/wal/instance
I20260923 15:06:01.715831 13614 external_mini_cluster.cc:1428] Setting key c2131fa69ef547da5dbaff962ef44997
I20260923 15:06:01.716615 2763 raft_consensus.cc:504] T 00000000000000000000000000000000 P 75a5aa3474fc4f85ad0ce399e61923df [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75a5aa3474fc4f85ad0ce399e61923df" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 41279 } }
I20260923 15:06:01.716753 2763 leader_election.cc:302] T 00000000000000000000000000000000 P 75a5aa3474fc4f85ad0ce399e61923df [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 75a5aa3474fc4f85ad0ce399e61923df; no voters:
I20260923 15:06:01.716929 2763 leader_election.cc:288] T 00000000000000000000000000000000 P 75a5aa3474fc4f85ad0ce399e61923df [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 15:06:01.717100 2769 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 75a5aa3474fc4f85ad0ce399e61923df [term 1 FOLLOWER]: Leader election won for term 1
I20260923 15:06:01.717240 2763 sys_catalog.cc:564] T 00000000000000000000000000000000 P 75a5aa3474fc4f85ad0ce399e61923df [sys.catalog]: configured and running, proceeding with master startup.
I20260923 15:06:01.717345 2769 raft_consensus.cc:686] T 00000000000000000000000000000000 P 75a5aa3474fc4f85ad0ce399e61923df [term 1 LEADER]: Becoming Leader. State: Replica: 75a5aa3474fc4f85ad0ce399e61923df, State: Running, Role: LEADER
I20260923 15:06:01.717486 2769 consensus_queue.cc:237] T 00000000000000000000000000000000 P 75a5aa3474fc4f85ad0ce399e61923df [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75a5aa3474fc4f85ad0ce399e61923df" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 41279 } }
I20260923 15:06:01.718688 2768 sys_catalog.cc:455] T 00000000000000000000000000000000 P 75a5aa3474fc4f85ad0ce399e61923df [sys.catalog]: SysCatalogTable state changed. Reason: New leader 75a5aa3474fc4f85ad0ce399e61923df. Latest consensus state: current_term: 1 leader_uuid: "75a5aa3474fc4f85ad0ce399e61923df" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75a5aa3474fc4f85ad0ce399e61923df" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 41279 } } }
I20260923 15:06:01.719148 2768 sys_catalog.cc:458] T 00000000000000000000000000000000 P 75a5aa3474fc4f85ad0ce399e61923df [sys.catalog]: This master's current role is: LEADER
I20260923 15:06:01.718868 2770 sys_catalog.cc:455] T 00000000000000000000000000000000 P 75a5aa3474fc4f85ad0ce399e61923df [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "75a5aa3474fc4f85ad0ce399e61923df" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "75a5aa3474fc4f85ad0ce399e61923df" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 41279 } } }
I20260923 15:06:01.719324 2770 sys_catalog.cc:458] T 00000000000000000000000000000000 P 75a5aa3474fc4f85ad0ce399e61923df [sys.catalog]: This master's current role is: LEADER
W20260923 15:06:01.719475 2783 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 75a5aa3474fc4f85ad0ce399e61923df: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260923 15:06:01.719558 2783 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260923 15:06:01.719974 2785 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 15:06:01.720189 2785 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 15:06:01.721880 2785 catalog_manager.cc:1347] Generated new cluster ID: 8ffc325c690e476188614f4db4a48898
I20260923 15:06:01.721936 2785 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
Sep 23 15:06:01 dist-test-slave-cnb0 krb5kdc[2681](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175961, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.75.190@KRBTEST.COM
I20260923 15:06:01.729175 2766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:01.716468 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40149 (local address 127.13.75.190:41279)
0923 15:06:01.716822 (+ 354us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:01.716833 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:01.716944 (+ 111us) server_negotiation.cc:407] Connection header received
0923 15:06:01.717220 (+ 276us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:01.717230 (+ 10us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:01.717301 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:01.717493 (+ 192us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:01.718375 (+ 882us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:01.719826 (+ 1451us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:01.720890 (+ 1064us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:01.721509 (+ 619us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:01.721632 (+ 123us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:01.724588 (+ 2956us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:06:01.724594 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:06:01.724594 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:06:01.724612 (+ 18us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:06:01.727291 (+ 2679us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:01.727794 (+ 503us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:01.727796 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:01.727797 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:01.727920 (+ 123us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:01.728181 (+ 261us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:01.728182 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:01.728182 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:01.728522 (+ 340us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:06:01.728687 (+ 165us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:01.728842 (+ 155us) server_negotiation.cc:299] Negotiation successful
0923 15:06:01.728944 (+ 102us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":287,"thread_start_us":122,"threads_started":1}
I20260923 15:06:01.730592 13614 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 15:06:01.739930 2766 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:01.732286 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:49407 (local address 127.13.75.190:41279)
0923 15:06:01.732323 (+ 37us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:01.732325 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:01.732334 (+ 9us) server_negotiation.cc:407] Connection header received
0923 15:06:01.732392 (+ 58us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:01.732393 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:01.732403 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:01.732454 (+ 51us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:01.733120 (+ 666us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:01.733854 (+ 734us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:01.734926 (+ 1072us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:01.735243 (+ 317us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:01.735306 (+ 63us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:01.736839 (+ 1533us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:06:01.736840 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:06:01.736841 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:06:01.736849 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:06:01.738447 (+ 1598us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:01.738933 (+ 486us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:01.738934 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:01.738935 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:01.738993 (+ 58us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:01.739246 (+ 253us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:01.739246 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:01.739247 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:01.739373 (+ 126us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:06:01.739463 (+ 90us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:01.739661 (+ 198us) server_negotiation.cc:299] Negotiation successful
0923 15:06:01.739726 (+ 65us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":17}
I20260923 15:06:01.755605 2785 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 15:06:01.756215 2785 catalog_manager.cc:1504] Loading token signing keys...
I20260923 15:06:01.770480 2785 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 75a5aa3474fc4f85ad0ce399e61923df: Generated new TSK 0
I20260923 15:06:01.770797 2785 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 15:06:01.773162 13614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKC4soy/build/release/bin/kudu with pid 2694
W20260923 15:06:01.777602 2679 connection.cc:535] client connection to 127.13.75.190:41279 recv error: Network error: failed to read from TLS socket (remote: 127.13.75.190:41279): Connection reset by peer (error 104)
2026-09-23T15:06:01Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/12 (1399 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/13
Loading random data
Initializing database '/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790175691491353-13614-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 15:06:01 dist-test-slave-cnb0 krb5kdc[2794](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 15:06:01 dist-test-slave-cnb0 krb5kdc[2794](info): set up 2 sockets
Sep 23 15:06:01 dist-test-slave-cnb0 krb5kdc[2794](info): commencing operation
krb5kdc: starting...
W20260923 15:06:02.828249 13614 mini_kdc.cc:121] Time spent starting KDC: real 1.037s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790175691491353-13614-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790175691491353-13614-0/krb5kdc/test-user.keytab.
Sep 23 15:06:02 dist-test-slave-cnb0 krb5kdc[2794](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175962, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T15:06:02Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T15:06:02Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.75.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.75.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.75.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.190.keytab.
Entry for principal kudu/127.13.75.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.190.keytab.
I20260923 15:06:02.958266 13614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKC4soy/build/release/bin/kudu
/tmp/dist-test-taskKC4soy/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790175691491353-13614-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790175691491353-13614-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790175691491353-13614-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790175691491353-13614-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.75.190:36831
--webserver_interface=127.13.75.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.190.keytab
--principal=kudu/127.13.75.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.75.148:39629
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.75.190:36831
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790175691491353-13614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790175691491353-13614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790175691491353-13614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:06:03.044269 2807 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:06:03.044518 2807 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:06:03.044545 2807 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:06:03.045800 2807 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 15:06:03.045847 2807 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:06:03.045862 2807 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:06:03.045876 2807 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 15:06:03.045888 2807 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 15:06:03.047516 2807 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.75.148:39629
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790175691491353-13614-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790175691491353-13614-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.75.190:36831
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.190.keytab
--principal=kudu/127.13.75.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.75.190:36831
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790175691491353-13614-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.75.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2807
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790175691491353-13614-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:06:03.047755 2807 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:06:03.047948 2807 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:06:03.053710 2813 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:06:03.053772 2807 server_base.cc:1034] running on GCE node
W20260923 15:06:03.053704 2812 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:06:03.053718 2815 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:06:03.054263 2807 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:06:03.054555 2807 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:06:03.055735 2807 hybrid_clock.cc:648] HybridClock initialized: now 1790175963055680 us; error 64 us; skew 500 ppm
Sep 23 15:06:03 dist-test-slave-cnb0 krb5kdc[2794](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175963, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:06:03.058775 2807 init.cc:377] Logged in from keytab as kudu/127.13.75.190@KRBTEST.COM (short username kudu)
I20260923 15:06:03.059333 2807 webserver.cc:460] Webserver started at http://127.13.75.190:37801/ using document root <none> and password file <none>
I20260923 15:06:03.059579 2807 fs_manager.cc:359] Metadata directory not provided
I20260923 15:06:03.059676 2807 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:06:03.059845 2807 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:06:03.060988 2807 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790175691491353-13614-0/minicluster-data/master-0/data/instance:
uuid: "54e2ca614b40468baea24a7616d944ff"
format_stamp: "Formatted at 2026-09-23 15:06:03 on dist-test-slave-cnb0"
server_key: "be26ff64e3522ff7ad29ee6113990089"
server_key_iv: "6ba550bb26928add7dd54d8eedc4fc02"
server_key_version: "encryptionkey@0"
I20260923 15:06:03.061399 2807 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790175691491353-13614-0/minicluster-data/master-0/wal/instance:
uuid: "54e2ca614b40468baea24a7616d944ff"
format_stamp: "Formatted at 2026-09-23 15:06:03 on dist-test-slave-cnb0"
server_key: "be26ff64e3522ff7ad29ee6113990089"
server_key_iv: "6ba550bb26928add7dd54d8eedc4fc02"
server_key_version: "encryptionkey@0"
I20260923 15:06:03.062870 2807 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260923 15:06:03.063903 2823 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:06:03.064127 2807 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260923 15:06:03.064235 2807 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790175691491353-13614-0/minicluster-data/master-0/data,/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790175691491353-13614-0/minicluster-data/master-0/wal
uuid: "54e2ca614b40468baea24a7616d944ff"
format_stamp: "Formatted at 2026-09-23 15:06:03 on dist-test-slave-cnb0"
server_key: "be26ff64e3522ff7ad29ee6113990089"
server_key_iv: "6ba550bb26928add7dd54d8eedc4fc02"
server_key_version: "encryptionkey@0"
I20260923 15:06:03.064316 2807 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790175691491353-13614-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790175691491353-13614-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790175691491353-13614-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:06:03.097213 2807 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:06:03.099736 2807 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:06:03.099922 2807 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:06:03.104036 2807 rpc_server.cc:307] RPC server started. Bound to: 127.13.75.190:36831
I20260923 15:06:03.104081 2875 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.75.190:36831 every 8 connection(s)
I20260923 15:06:03.104514 2807 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790175691491353-13614-0/minicluster-data/master-0/data/info.pb
I20260923 15:06:03.105520 2876 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:06:03.107968 2876 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 54e2ca614b40468baea24a7616d944ff: Bootstrap starting.
I20260923 15:06:03.108588 2876 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 54e2ca614b40468baea24a7616d944ff: Neither blocks nor log segments found. Creating new log.
I20260923 15:06:03.108898 2876 log.cc:824] T 00000000000000000000000000000000 P 54e2ca614b40468baea24a7616d944ff: Log is configured to *not* fsync() on all Append() calls
I20260923 15:06:03.109675 2876 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 54e2ca614b40468baea24a7616d944ff: No bootstrap required, opened a new log
I20260923 15:06:03.111850 2876 raft_consensus.cc:348] T 00000000000000000000000000000000 P 54e2ca614b40468baea24a7616d944ff [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54e2ca614b40468baea24a7616d944ff" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 36831 } }
I20260923 15:06:03.112052 2876 raft_consensus.cc:374] T 00000000000000000000000000000000 P 54e2ca614b40468baea24a7616d944ff [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:06:03.112105 2876 raft_consensus.cc:729] T 00000000000000000000000000000000 P 54e2ca614b40468baea24a7616d944ff [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 54e2ca614b40468baea24a7616d944ff, State: Initialized, Role: FOLLOWER
I20260923 15:06:03.112324 2876 consensus_queue.cc:260] T 00000000000000000000000000000000 P 54e2ca614b40468baea24a7616d944ff [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54e2ca614b40468baea24a7616d944ff" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 36831 } }
I20260923 15:06:03.112427 2876 raft_consensus.cc:388] T 00000000000000000000000000000000 P 54e2ca614b40468baea24a7616d944ff [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 15:06:03.112476 2876 raft_consensus.cc:482] T 00000000000000000000000000000000 P 54e2ca614b40468baea24a7616d944ff [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 15:06:03.112536 2876 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 54e2ca614b40468baea24a7616d944ff [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:06:03.113556 2876 raft_consensus.cc:504] T 00000000000000000000000000000000 P 54e2ca614b40468baea24a7616d944ff [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54e2ca614b40468baea24a7616d944ff" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 36831 } }
I20260923 15:06:03.113703 2876 leader_election.cc:302] T 00000000000000000000000000000000 P 54e2ca614b40468baea24a7616d944ff [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 54e2ca614b40468baea24a7616d944ff; no voters:
I20260923 15:06:03.113917 2876 leader_election.cc:288] T 00000000000000000000000000000000 P 54e2ca614b40468baea24a7616d944ff [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 15:06:03.113955 2879 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 54e2ca614b40468baea24a7616d944ff [term 1 FOLLOWER]: Leader election won for term 1
I20260923 15:06:03.114225 2876 sys_catalog.cc:564] T 00000000000000000000000000000000 P 54e2ca614b40468baea24a7616d944ff [sys.catalog]: configured and running, proceeding with master startup.
I20260923 15:06:03.114359 13614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKC4soy/build/release/bin/kudu as pid 2807
I20260923 15:06:03.114480 13614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_13.1790175691491353-13614-0/minicluster-data/master-0/wal/instance
I20260923 15:06:03.114666 13614 external_mini_cluster.cc:1428] Setting key 940cd54ec97805dd8703c44b39b32aa3
I20260923 15:06:03.114838 2879 raft_consensus.cc:686] T 00000000000000000000000000000000 P 54e2ca614b40468baea24a7616d944ff [term 1 LEADER]: Becoming Leader. State: Replica: 54e2ca614b40468baea24a7616d944ff, State: Running, Role: LEADER
I20260923 15:06:03.114964 2879 consensus_queue.cc:237] T 00000000000000000000000000000000 P 54e2ca614b40468baea24a7616d944ff [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54e2ca614b40468baea24a7616d944ff" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 36831 } }
I20260923 15:06:03.115662 2881 sys_catalog.cc:455] T 00000000000000000000000000000000 P 54e2ca614b40468baea24a7616d944ff [sys.catalog]: SysCatalogTable state changed. Reason: New leader 54e2ca614b40468baea24a7616d944ff. Latest consensus state: current_term: 1 leader_uuid: "54e2ca614b40468baea24a7616d944ff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54e2ca614b40468baea24a7616d944ff" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 36831 } } }
I20260923 15:06:03.115777 2881 sys_catalog.cc:458] T 00000000000000000000000000000000 P 54e2ca614b40468baea24a7616d944ff [sys.catalog]: This master's current role is: LEADER
I20260923 15:06:03.116014 2880 sys_catalog.cc:455] T 00000000000000000000000000000000 P 54e2ca614b40468baea24a7616d944ff [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "54e2ca614b40468baea24a7616d944ff" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "54e2ca614b40468baea24a7616d944ff" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 36831 } } }
I20260923 15:06:03.116117 2880 sys_catalog.cc:458] T 00000000000000000000000000000000 P 54e2ca614b40468baea24a7616d944ff [sys.catalog]: This master's current role is: LEADER
W20260923 15:06:03.116328 2895 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 54e2ca614b40468baea24a7616d944ff: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260923 15:06:03.116389 2895 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260923 15:06:03.116667 2896 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 15:06:03.116873 2896 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 15:06:03.119175 2896 catalog_manager.cc:1347] Generated new cluster ID: e7c5927e271e4592abd68741cf2dbc55
I20260923 15:06:03.119239 2896 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 15:06:03.121707 2894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:03.115568 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34817 (local address 127.13.75.190:36831)
0923 15:06:03.115761 (+ 193us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:03.115857 (+ 96us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:03.115899 (+ 42us) server_negotiation.cc:407] Connection header received
0923 15:06:03.116237 (+ 338us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:03.116246 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:03.116315 (+ 69us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:03.116734 (+ 419us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:03.116836 (+ 102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:03.118062 (+ 1226us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:03.119110 (+ 1048us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:03.119717 (+ 607us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:03.119820 (+ 103us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:03.120841 (+ 1021us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:06:03.120845 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:06:03.120846 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 15:06:03.121100 (+ 254us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:06:03.121147 (+ 47us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 15:06:03.121176 (+ 29us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:06:03.121291 (+ 115us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:03.121419 (+ 128us) server_negotiation.cc:299] Negotiation successful
0923 15:06:03.121456 (+ 37us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":132,"thread_start_us":86,"threads_started":1}
I20260923 15:06:03.123313 13614 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 15:06:03.129380 2894 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:03.125370 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:56871 (local address 127.13.75.190:36831)
0923 15:06:03.125424 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:03.125427 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:03.125437 (+ 10us) server_negotiation.cc:407] Connection header received
0923 15:06:03.125466 (+ 29us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:03.125468 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:03.125479 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:03.125552 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:03.125732 (+ 180us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:03.126606 (+ 874us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:03.127719 (+ 1113us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:03.128123 (+ 404us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:03.128225 (+ 102us) server_negotiation.cc:662] Negotiated TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:03.129054 (+ 829us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:06:03.129055 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:06:03.129056 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 15:06:03.129063 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:06:03.129083 (+ 20us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 15:06:03.129089 (+ 6us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:06:03.129167 (+ 78us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:03.129256 (+ 89us) server_negotiation.cc:299] Negotiation successful
0923 15:06:03.129276 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260923 15:06:03.143599 2896 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 15:06:03.144470 2896 catalog_manager.cc:1504] Loading token signing keys...
I20260923 15:06:03.154196 2896 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 54e2ca614b40468baea24a7616d944ff: Generated new TSK 0
I20260923 15:06:03.154470 2896 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 15:06:03.162494 13614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKC4soy/build/release/bin/kudu with pid 2807
W20260923 15:06:03.167321 2792 connection.cc:535] client connection to 127.13.75.190:36831 recv error: Network error: failed to read from TLS socket (remote: 127.13.75.190:36831): Connection reset by peer (error 104)
2026-09-23T15:06:03Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/13 (1389 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/14
Loading random data
Initializing database '/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790175691491353-13614-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 15:06:03 dist-test-slave-cnb0 krb5kdc[2907](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 15:06:03 dist-test-slave-cnb0 krb5kdc[2907](info): set up 2 sockets
Sep 23 15:06:03 dist-test-slave-cnb0 krb5kdc[2907](info): commencing operation
krb5kdc: starting...
W20260923 15:06:04.220624 13614 mini_kdc.cc:121] Time spent starting KDC: real 1.039s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790175691491353-13614-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790175691491353-13614-0/krb5kdc/test-user.keytab.
Sep 23 15:06:04 dist-test-slave-cnb0 krb5kdc[2907](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175964, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T15:06:04Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T15:06:04Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.75.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.75.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.75.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.190.keytab.
Entry for principal kudu/127.13.75.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.190.keytab.
I20260923 15:06:04.350620 13614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKC4soy/build/release/bin/kudu
/tmp/dist-test-taskKC4soy/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790175691491353-13614-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790175691491353-13614-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790175691491353-13614-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790175691491353-13614-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.75.190:40919
--webserver_interface=127.13.75.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.190.keytab
--principal=kudu/127.13.75.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.75.148:41717
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.75.190:40919
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=true with env {KRB5CCNAME=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790175691491353-13614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790175691491353-13614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790175691491353-13614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:06:04.434751 2920 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:06:04.434999 2920 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:06:04.435026 2920 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:06:04.436177 2920 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 15:06:04.436214 2920 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:06:04.436228 2920 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:06:04.436241 2920 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 15:06:04.436254 2920 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 15:06:04.437749 2920 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.75.148:41717
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790175691491353-13614-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790175691491353-13614-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.75.190:40919
--rpc_encrypt_loopback_connections=true
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.190.keytab
--principal=kudu/127.13.75.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.75.190:40919
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790175691491353-13614-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.75.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.2920
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790175691491353-13614-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:06:04.437965 2920 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:06:04.438165 2920 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:06:04.443511 2925 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:06:04.443656 2920 server_base.cc:1034] running on GCE node
W20260923 15:06:04.443508 2926 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:06:04.443773 2928 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:06:04.444116 2920 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:06:04.444424 2920 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:06:04.445559 2920 hybrid_clock.cc:648] HybridClock initialized: now 1790175964445534 us; error 46 us; skew 500 ppm
Sep 23 15:06:04 dist-test-slave-cnb0 krb5kdc[2907](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175964, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:06:04.448426 2920 init.cc:377] Logged in from keytab as kudu/127.13.75.190@KRBTEST.COM (short username kudu)
I20260923 15:06:04.448984 2920 webserver.cc:460] Webserver started at http://127.13.75.190:46353/ using document root <none> and password file <none>
I20260923 15:06:04.449338 2920 fs_manager.cc:359] Metadata directory not provided
I20260923 15:06:04.449426 2920 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:06:04.449586 2920 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:06:04.450714 2920 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790175691491353-13614-0/minicluster-data/master-0/data/instance:
uuid: "c749621411874f37bc4b5990e375286e"
format_stamp: "Formatted at 2026-09-23 15:06:04 on dist-test-slave-cnb0"
server_key: "f40ac76822fdf71b4d2109554ae193bb"
server_key_iv: "1e2e89e4064e650f8f6a5eb1aea487c9"
server_key_version: "encryptionkey@0"
I20260923 15:06:04.451162 2920 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790175691491353-13614-0/minicluster-data/master-0/wal/instance:
uuid: "c749621411874f37bc4b5990e375286e"
format_stamp: "Formatted at 2026-09-23 15:06:04 on dist-test-slave-cnb0"
server_key: "f40ac76822fdf71b4d2109554ae193bb"
server_key_iv: "1e2e89e4064e650f8f6a5eb1aea487c9"
server_key_version: "encryptionkey@0"
I20260923 15:06:04.452616 2920 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:06:04.453481 2935 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:06:04.453840 2920 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:06:04.453960 2920 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790175691491353-13614-0/minicluster-data/master-0/data,/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790175691491353-13614-0/minicluster-data/master-0/wal
uuid: "c749621411874f37bc4b5990e375286e"
format_stamp: "Formatted at 2026-09-23 15:06:04 on dist-test-slave-cnb0"
server_key: "f40ac76822fdf71b4d2109554ae193bb"
server_key_iv: "1e2e89e4064e650f8f6a5eb1aea487c9"
server_key_version: "encryptionkey@0"
I20260923 15:06:04.454047 2920 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790175691491353-13614-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790175691491353-13614-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790175691491353-13614-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:06:04.457659 2920 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:06:04.459944 2920 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:06:04.460114 2920 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:06:04.464334 2920 rpc_server.cc:307] RPC server started. Bound to: 127.13.75.190:40919
I20260923 15:06:04.464414 2989 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.75.190:40919 every 8 connection(s)
I20260923 15:06:04.464775 2920 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790175691491353-13614-0/minicluster-data/master-0/data/info.pb
I20260923 15:06:04.465895 2990 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:06:04.465904 13614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKC4soy/build/release/bin/kudu as pid 2920
I20260923 15:06:04.466043 13614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_14.1790175691491353-13614-0/minicluster-data/master-0/wal/instance
I20260923 15:06:04.466250 13614 external_mini_cluster.cc:1428] Setting key de20ed4208d7dd31670b237f60cbb991
I20260923 15:06:04.469262 2993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:04.467005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:38579 (local address 127.13.75.190:40919)
0923 15:06:04.467231 (+ 226us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:04.467241 (+ 10us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:04.467270 (+ 29us) server_negotiation.cc:407] Connection header received
0923 15:06:04.467782 (+ 512us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:04.467791 (+ 9us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:04.467822 (+ 31us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:04.467999 (+ 177us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:04.468761 (+ 762us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:06:04.468765 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:06:04.468766 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 15:06:04.468770 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:06:04.468810 (+ 40us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 15:06:04.468822 (+ 12us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:06:04.468902 (+ 80us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:04.468984 (+ 82us) server_negotiation.cc:299] Negotiation successful
0923 15:06:04.469031 (+ 47us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":141,"thread_start_us":76,"threads_started":1}
I20260923 15:06:04.469650 2990 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c749621411874f37bc4b5990e375286e: Bootstrap starting.
I20260923 15:06:04.470350 13614 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 15:06:04.470492 2990 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c749621411874f37bc4b5990e375286e: Neither blocks nor log segments found. Creating new log.
I20260923 15:06:04.470821 2990 log.cc:824] T 00000000000000000000000000000000 P c749621411874f37bc4b5990e375286e: Log is configured to *not* fsync() on all Append() calls
I20260923 15:06:04.471866 2990 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c749621411874f37bc4b5990e375286e: No bootstrap required, opened a new log
I20260923 15:06:04.473152 2993 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:04.472050 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:34161 (local address 127.13.75.190:40919)
0923 15:06:04.472112 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:04.472114 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:04.472274 (+ 160us) server_negotiation.cc:407] Connection header received
0923 15:06:04.472349 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:04.472350 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:04.472355 (+ 5us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:04.472403 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:04.472966 (+ 563us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:06:04.472967 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:06:04.472967 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 15:06:04.472971 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:06:04.472983 (+ 12us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 15:06:04.472987 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:06:04.473036 (+ 49us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:04.473094 (+ 58us) server_negotiation.cc:299] Negotiation successful
0923 15:06:04.473105 (+ 11us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":39}
I20260923 15:06:04.474516 2990 raft_consensus.cc:348] T 00000000000000000000000000000000 P c749621411874f37bc4b5990e375286e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c749621411874f37bc4b5990e375286e" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 40919 } }
I20260923 15:06:04.474691 2990 raft_consensus.cc:374] T 00000000000000000000000000000000 P c749621411874f37bc4b5990e375286e [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:06:04.474740 2990 raft_consensus.cc:729] T 00000000000000000000000000000000 P c749621411874f37bc4b5990e375286e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c749621411874f37bc4b5990e375286e, State: Initialized, Role: FOLLOWER
I20260923 15:06:04.474874 2990 consensus_queue.cc:260] T 00000000000000000000000000000000 P c749621411874f37bc4b5990e375286e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c749621411874f37bc4b5990e375286e" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 40919 } }
I20260923 15:06:04.475030 2990 raft_consensus.cc:388] T 00000000000000000000000000000000 P c749621411874f37bc4b5990e375286e [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 15:06:04.475078 2990 raft_consensus.cc:482] T 00000000000000000000000000000000 P c749621411874f37bc4b5990e375286e [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 15:06:04.475131 2990 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c749621411874f37bc4b5990e375286e [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:06:04.475755 2990 raft_consensus.cc:504] T 00000000000000000000000000000000 P c749621411874f37bc4b5990e375286e [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c749621411874f37bc4b5990e375286e" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 40919 } }
I20260923 15:06:04.475878 2990 leader_election.cc:302] T 00000000000000000000000000000000 P c749621411874f37bc4b5990e375286e [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c749621411874f37bc4b5990e375286e; no voters:
I20260923 15:06:04.476078 2990 leader_election.cc:288] T 00000000000000000000000000000000 P c749621411874f37bc4b5990e375286e [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 15:06:04.476125 3000 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c749621411874f37bc4b5990e375286e [term 1 FOLLOWER]: Leader election won for term 1
I20260923 15:06:04.476325 3000 raft_consensus.cc:686] T 00000000000000000000000000000000 P c749621411874f37bc4b5990e375286e [term 1 LEADER]: Becoming Leader. State: Replica: c749621411874f37bc4b5990e375286e, State: Running, Role: LEADER
I20260923 15:06:04.476351 2990 sys_catalog.cc:564] T 00000000000000000000000000000000 P c749621411874f37bc4b5990e375286e [sys.catalog]: configured and running, proceeding with master startup.
I20260923 15:06:04.476423 3000 consensus_queue.cc:237] T 00000000000000000000000000000000 P c749621411874f37bc4b5990e375286e [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c749621411874f37bc4b5990e375286e" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 40919 } }
I20260923 15:06:04.476824 3002 sys_catalog.cc:455] T 00000000000000000000000000000000 P c749621411874f37bc4b5990e375286e [sys.catalog]: SysCatalogTable state changed. Reason: New leader c749621411874f37bc4b5990e375286e. Latest consensus state: current_term: 1 leader_uuid: "c749621411874f37bc4b5990e375286e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c749621411874f37bc4b5990e375286e" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 40919 } } }
I20260923 15:06:04.476946 3002 sys_catalog.cc:458] T 00000000000000000000000000000000 P c749621411874f37bc4b5990e375286e [sys.catalog]: This master's current role is: LEADER
I20260923 15:06:04.476821 3001 sys_catalog.cc:455] T 00000000000000000000000000000000 P c749621411874f37bc4b5990e375286e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c749621411874f37bc4b5990e375286e" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c749621411874f37bc4b5990e375286e" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 40919 } } }
I20260923 15:06:04.477111 3001 sys_catalog.cc:458] T 00000000000000000000000000000000 P c749621411874f37bc4b5990e375286e [sys.catalog]: This master's current role is: LEADER
I20260923 15:06:04.477228 3009 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 15:06:04.477701 3009 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 15:06:04.479401 3009 catalog_manager.cc:1347] Generated new cluster ID: ceca3269f39a4d9da7092de0994042cb
I20260923 15:06:04.479455 3009 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 15:06:04.485453 3009 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 15:06:04.485641 3009 catalog_manager.cc:1504] Loading token signing keys...
I20260923 15:06:04.490049 3009 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c749621411874f37bc4b5990e375286e: Generated new TSK 0
I20260923 15:06:04.490288 3009 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 15:06:04.496825 13614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKC4soy/build/release/bin/kudu with pid 2920
2026-09-23T15:06:04Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/14 (1334 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/15
Loading random data
Initializing database '/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790175691491353-13614-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 15:06:04 dist-test-slave-cnb0 krb5kdc[3021](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 15:06:04 dist-test-slave-cnb0 krb5kdc[3021](info): set up 2 sockets
Sep 23 15:06:04 dist-test-slave-cnb0 krb5kdc[3021](info): commencing operation
krb5kdc: starting...
W20260923 15:06:05.560504 13614 mini_kdc.cc:121] Time spent starting KDC: real 1.045s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790175691491353-13614-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790175691491353-13614-0/krb5kdc/test-user.keytab.
Sep 23 15:06:05 dist-test-slave-cnb0 krb5kdc[3021](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175965, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T15:06:05Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T15:06:05Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.75.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.75.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.75.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.190.keytab.
Entry for principal kudu/127.13.75.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.190.keytab.
I20260923 15:06:05.692692 13614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKC4soy/build/release/bin/kudu
/tmp/dist-test-taskKC4soy/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790175691491353-13614-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790175691491353-13614-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790175691491353-13614-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790175691491353-13614-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.75.190:37453
--webserver_interface=127.13.75.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.190.keytab
--principal=kudu/127.13.75.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.75.148:32835
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.75.190:37453
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=required
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790175691491353-13614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790175691491353-13614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790175691491353-13614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:06:05.781033 3034 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:06:05.781344 3034 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:06:05.781391 3034 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:06:05.782763 3034 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 15:06:05.782855 3034 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:06:05.782877 3034 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:06:05.782896 3034 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 15:06:05.782945 3034 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 15:06:05.784864 3034 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.75.148:32835
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790175691491353-13614-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790175691491353-13614-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.75.190:37453
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.190.keytab
--principal=kudu/127.13.75.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.75.190:37453
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790175691491353-13614-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.75.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3034
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790175691491353-13614-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:06:05.785219 3034 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:06:05.785494 3034 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260923 15:06:05.791625 3034 server_base.cc:1034] running on GCE node
W20260923 15:06:05.791606 3040 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:06:05.791627 3039 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:06:05.791606 3042 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:06:05.792333 3034 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:06:05.792681 3034 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:06:05.793869 3034 hybrid_clock.cc:648] HybridClock initialized: now 1790175965793844 us; error 39 us; skew 500 ppm
Sep 23 15:06:05 dist-test-slave-cnb0 krb5kdc[3021](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175965, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:06:05.796662 3034 init.cc:377] Logged in from keytab as kudu/127.13.75.190@KRBTEST.COM (short username kudu)
I20260923 15:06:05.797189 3034 webserver.cc:460] Webserver started at http://127.13.75.190:38627/ using document root <none> and password file <none>
I20260923 15:06:05.797456 3034 fs_manager.cc:359] Metadata directory not provided
I20260923 15:06:05.797528 3034 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:06:05.797677 3034 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:06:05.798866 3034 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790175691491353-13614-0/minicluster-data/master-0/data/instance:
uuid: "c42052c9c8ee46c0bd81d4069cfb1730"
format_stamp: "Formatted at 2026-09-23 15:06:05 on dist-test-slave-cnb0"
server_key: "d512c62d4b9b3bd1373cc646ac690376"
server_key_iv: "c2c7bce16a18db7957267c1cc5628079"
server_key_version: "encryptionkey@0"
I20260923 15:06:05.799329 3034 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790175691491353-13614-0/minicluster-data/master-0/wal/instance:
uuid: "c42052c9c8ee46c0bd81d4069cfb1730"
format_stamp: "Formatted at 2026-09-23 15:06:05 on dist-test-slave-cnb0"
server_key: "d512c62d4b9b3bd1373cc646ac690376"
server_key_iv: "c2c7bce16a18db7957267c1cc5628079"
server_key_version: "encryptionkey@0"
I20260923 15:06:05.800897 3034 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.002s
I20260923 15:06:05.801906 3049 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:06:05.802198 3034 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260923 15:06:05.802398 3034 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790175691491353-13614-0/minicluster-data/master-0/data,/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790175691491353-13614-0/minicluster-data/master-0/wal
uuid: "c42052c9c8ee46c0bd81d4069cfb1730"
format_stamp: "Formatted at 2026-09-23 15:06:05 on dist-test-slave-cnb0"
server_key: "d512c62d4b9b3bd1373cc646ac690376"
server_key_iv: "c2c7bce16a18db7957267c1cc5628079"
server_key_version: "encryptionkey@0"
I20260923 15:06:05.802515 3034 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790175691491353-13614-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790175691491353-13614-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790175691491353-13614-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:06:05.827718 3034 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:06:05.830152 3034 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:06:05.830351 3034 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:06:05.834587 3034 rpc_server.cc:307] RPC server started. Bound to: 127.13.75.190:37453
I20260923 15:06:05.834627 3102 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.75.190:37453 every 8 connection(s)
I20260923 15:06:05.835037 3034 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790175691491353-13614-0/minicluster-data/master-0/data/info.pb
I20260923 15:06:05.836047 3103 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:06:05.838375 3103 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c42052c9c8ee46c0bd81d4069cfb1730: Bootstrap starting.
I20260923 15:06:05.839107 3103 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P c42052c9c8ee46c0bd81d4069cfb1730: Neither blocks nor log segments found. Creating new log.
I20260923 15:06:05.839184 13614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKC4soy/build/release/bin/kudu as pid 3034
I20260923 15:06:05.839294 13614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_15.1790175691491353-13614-0/minicluster-data/master-0/wal/instance
I20260923 15:06:05.839479 3103 log.cc:824] T 00000000000000000000000000000000 P c42052c9c8ee46c0bd81d4069cfb1730: Log is configured to *not* fsync() on all Append() calls
I20260923 15:06:05.839504 13614 external_mini_cluster.cc:1428] Setting key ff38ec0761b111fb1d16ec6c8643295c
I20260923 15:06:05.840328 3103 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P c42052c9c8ee46c0bd81d4069cfb1730: No bootstrap required, opened a new log
I20260923 15:06:05.843037 3103 raft_consensus.cc:348] T 00000000000000000000000000000000 P c42052c9c8ee46c0bd81d4069cfb1730 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c42052c9c8ee46c0bd81d4069cfb1730" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 37453 } }
I20260923 15:06:05.843225 3103 raft_consensus.cc:374] T 00000000000000000000000000000000 P c42052c9c8ee46c0bd81d4069cfb1730 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:06:05.843261 3103 raft_consensus.cc:729] T 00000000000000000000000000000000 P c42052c9c8ee46c0bd81d4069cfb1730 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: c42052c9c8ee46c0bd81d4069cfb1730, State: Initialized, Role: FOLLOWER
I20260923 15:06:05.843386 3103 consensus_queue.cc:260] T 00000000000000000000000000000000 P c42052c9c8ee46c0bd81d4069cfb1730 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c42052c9c8ee46c0bd81d4069cfb1730" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 37453 } }
I20260923 15:06:05.843462 3103 raft_consensus.cc:388] T 00000000000000000000000000000000 P c42052c9c8ee46c0bd81d4069cfb1730 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 15:06:05.843493 3103 raft_consensus.cc:482] T 00000000000000000000000000000000 P c42052c9c8ee46c0bd81d4069cfb1730 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 15:06:05.843545 3103 raft_consensus.cc:3037] T 00000000000000000000000000000000 P c42052c9c8ee46c0bd81d4069cfb1730 [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:06:05.844254 3103 raft_consensus.cc:504] T 00000000000000000000000000000000 P c42052c9c8ee46c0bd81d4069cfb1730 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c42052c9c8ee46c0bd81d4069cfb1730" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 37453 } }
I20260923 15:06:05.844344 3103 leader_election.cc:302] T 00000000000000000000000000000000 P c42052c9c8ee46c0bd81d4069cfb1730 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: c42052c9c8ee46c0bd81d4069cfb1730; no voters:
I20260923 15:06:05.844555 3103 leader_election.cc:288] T 00000000000000000000000000000000 P c42052c9c8ee46c0bd81d4069cfb1730 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 15:06:05.844579 3108 raft_consensus.cc:2781] T 00000000000000000000000000000000 P c42052c9c8ee46c0bd81d4069cfb1730 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 15:06:05.844861 3108 raft_consensus.cc:686] T 00000000000000000000000000000000 P c42052c9c8ee46c0bd81d4069cfb1730 [term 1 LEADER]: Becoming Leader. State: Replica: c42052c9c8ee46c0bd81d4069cfb1730, State: Running, Role: LEADER
I20260923 15:06:05.845010 3103 sys_catalog.cc:564] T 00000000000000000000000000000000 P c42052c9c8ee46c0bd81d4069cfb1730 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 15:06:05.845177 3108 consensus_queue.cc:237] T 00000000000000000000000000000000 P c42052c9c8ee46c0bd81d4069cfb1730 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c42052c9c8ee46c0bd81d4069cfb1730" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 37453 } }
I20260923 15:06:05.845542 3108 sys_catalog.cc:455] T 00000000000000000000000000000000 P c42052c9c8ee46c0bd81d4069cfb1730 [sys.catalog]: SysCatalogTable state changed. Reason: New leader c42052c9c8ee46c0bd81d4069cfb1730. Latest consensus state: current_term: 1 leader_uuid: "c42052c9c8ee46c0bd81d4069cfb1730" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c42052c9c8ee46c0bd81d4069cfb1730" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 37453 } } }
I20260923 15:06:05.845611 3108 sys_catalog.cc:458] T 00000000000000000000000000000000 P c42052c9c8ee46c0bd81d4069cfb1730 [sys.catalog]: This master's current role is: LEADER
I20260923 15:06:05.845752 3109 sys_catalog.cc:455] T 00000000000000000000000000000000 P c42052c9c8ee46c0bd81d4069cfb1730 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "c42052c9c8ee46c0bd81d4069cfb1730" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "c42052c9c8ee46c0bd81d4069cfb1730" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 37453 } } }
I20260923 15:06:05.845841 3109 sys_catalog.cc:458] T 00000000000000000000000000000000 P c42052c9c8ee46c0bd81d4069cfb1730 [sys.catalog]: This master's current role is: LEADER
I20260923 15:06:05.846661 3116 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
Sep 23 15:06:05 dist-test-slave-cnb0 krb5kdc[3021](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175965, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.75.190@KRBTEST.COM
I20260923 15:06:05.847734 3116 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 15:06:05.849429 3116 catalog_manager.cc:1347] Generated new cluster ID: 8370418f013e4bc7885d406a1660c46e
I20260923 15:06:05.849485 3116 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 15:06:05.852119 3107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:05.840131 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:46593 (local address 127.13.75.190:37453)
0923 15:06:05.840311 (+ 180us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:05.840319 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:05.840348 (+ 29us) server_negotiation.cc:407] Connection header received
0923 15:06:05.840849 (+ 501us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:05.840857 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:05.840909 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:05.841035 (+ 126us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:05.841883 (+ 848us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:05.842830 (+ 947us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:05.843988 (+ 1158us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:05.844835 (+ 847us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:05.845345 (+ 510us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:05.848122 (+ 2777us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:06:05.848126 (+ 4us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:06:05.848127 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:06:05.848141 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:06:05.850286 (+ 2145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:05.850716 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:05.850719 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:05.850720 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:05.850851 (+ 131us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:05.851114 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:05.851115 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:05.851116 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:05.851446 (+ 330us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:06:05.851620 (+ 174us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:05.851719 (+ 99us) server_negotiation.cc:299] Negotiation successful
0923 15:06:05.851883 (+ 164us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":120,"thread_start_us":88,"threads_started":1}
I20260923 15:06:05.853097 13614 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 15:06:05.867383 3107 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:05.855235 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44469 (local address 127.13.75.190:37453)
0923 15:06:05.855270 (+ 35us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:05.855272 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:05.859224 (+ 3952us) server_negotiation.cc:407] Connection header received
0923 15:06:05.859356 (+ 132us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:05.859358 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:05.859373 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:05.859450 (+ 77us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:05.860195 (+ 745us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:05.861070 (+ 875us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:05.862079 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:05.862470 (+ 391us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:05.862552 (+ 82us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:05.864147 (+ 1595us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:06:05.864149 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:06:05.864149 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:06:05.864158 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:06:05.865993 (+ 1835us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:05.866406 (+ 413us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:05.866407 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:05.866407 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:05.866503 (+ 96us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:05.866721 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:05.866721 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:05.866722 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:05.866879 (+ 157us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:06:05.867027 (+ 148us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:05.867176 (+ 149us) server_negotiation.cc:299] Negotiation successful
0923 15:06:05.867270 (+ 94us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":16}
I20260923 15:06:05.878903 3116 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 15:06:05.879822 3116 catalog_manager.cc:1504] Loading token signing keys...
I20260923 15:06:05.888893 3116 catalog_manager.cc:5868] T 00000000000000000000000000000000 P c42052c9c8ee46c0bd81d4069cfb1730: Generated new TSK 0
I20260923 15:06:05.889138 3116 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 15:06:05.900346 13614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKC4soy/build/release/bin/kudu with pid 3034
2026-09-23T15:06:05Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/15 (1403 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/16
Loading random data
Initializing database '/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790175691491353-13614-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 15:06:05 dist-test-slave-cnb0 krb5kdc[3134](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 15:06:05 dist-test-slave-cnb0 krb5kdc[3134](info): set up 2 sockets
Sep 23 15:06:05 dist-test-slave-cnb0 krb5kdc[3134](info): commencing operation
krb5kdc: starting...
W20260923 15:06:06.984290 13614 mini_kdc.cc:121] Time spent starting KDC: real 1.065s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790175691491353-13614-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790175691491353-13614-0/krb5kdc/test-user.keytab.
Sep 23 15:06:07 dist-test-slave-cnb0 krb5kdc[3134](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175967, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T15:06:07Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T15:06:07Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.75.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.75.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.75.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.190.keytab.
Entry for principal kudu/127.13.75.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.190.keytab.
I20260923 15:06:07.114727 13614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKC4soy/build/release/bin/kudu
/tmp/dist-test-taskKC4soy/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790175691491353-13614-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790175691491353-13614-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790175691491353-13614-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790175691491353-13614-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.75.190:44169
--webserver_interface=127.13.75.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.190.keytab
--principal=kudu/127.13.75.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.75.148:46675
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.75.190:44169
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=required
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790175691491353-13614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790175691491353-13614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790175691491353-13614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:06:07.198326 3147 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:06:07.198567 3147 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:06:07.198594 3147 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:06:07.199890 3147 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 15:06:07.199946 3147 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:06:07.199961 3147 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:06:07.199975 3147 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 15:06:07.199987 3147 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 15:06:07.201509 3147 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.75.148:46675
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790175691491353-13614-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790175691491353-13614-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.75.190:44169
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.190.keytab
--principal=kudu/127.13.75.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.75.190:44169
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=required
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790175691491353-13614-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.75.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3147
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790175691491353-13614-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:06:07.201730 3147 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:06:07.201916 3147 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:06:07.206960 3155 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:06:07.207146 3147 server_base.cc:1034] running on GCE node
W20260923 15:06:07.206966 3152 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:06:07.206977 3153 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:06:07.207517 3147 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:06:07.207845 3147 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:06:07.209017 3147 hybrid_clock.cc:648] HybridClock initialized: now 1790175967208981 us; error 55 us; skew 500 ppm
Sep 23 15:06:07 dist-test-slave-cnb0 krb5kdc[3134](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175967, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:06:07.211877 3147 init.cc:377] Logged in from keytab as kudu/127.13.75.190@KRBTEST.COM (short username kudu)
I20260923 15:06:07.212392 3147 webserver.cc:460] Webserver started at http://127.13.75.190:42021/ using document root <none> and password file <none>
I20260923 15:06:07.212639 3147 fs_manager.cc:359] Metadata directory not provided
I20260923 15:06:07.212714 3147 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:06:07.212888 3147 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:06:07.213999 3147 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790175691491353-13614-0/minicluster-data/master-0/data/instance:
uuid: "88742e54456d49f99af7b84a809b8678"
format_stamp: "Formatted at 2026-09-23 15:06:07 on dist-test-slave-cnb0"
server_key: "cecd9d3975a5914ad4ca6ecc08bf009a"
server_key_iv: "045c0521ef10ff9b60923b2e071d5b38"
server_key_version: "encryptionkey@0"
I20260923 15:06:07.214381 3147 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790175691491353-13614-0/minicluster-data/master-0/wal/instance:
uuid: "88742e54456d49f99af7b84a809b8678"
format_stamp: "Formatted at 2026-09-23 15:06:07 on dist-test-slave-cnb0"
server_key: "cecd9d3975a5914ad4ca6ecc08bf009a"
server_key_iv: "045c0521ef10ff9b60923b2e071d5b38"
server_key_version: "encryptionkey@0"
I20260923 15:06:07.215939 3147 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260923 15:06:07.216830 3162 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:06:07.217063 3147 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.000s sys 0.001s
I20260923 15:06:07.217175 3147 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790175691491353-13614-0/minicluster-data/master-0/data,/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790175691491353-13614-0/minicluster-data/master-0/wal
uuid: "88742e54456d49f99af7b84a809b8678"
format_stamp: "Formatted at 2026-09-23 15:06:07 on dist-test-slave-cnb0"
server_key: "cecd9d3975a5914ad4ca6ecc08bf009a"
server_key_iv: "045c0521ef10ff9b60923b2e071d5b38"
server_key_version: "encryptionkey@0"
I20260923 15:06:07.217275 3147 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790175691491353-13614-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790175691491353-13614-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790175691491353-13614-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:06:07.254191 3147 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:06:07.256623 3147 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:06:07.256796 3147 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:06:07.261220 3147 rpc_server.cc:307] RPC server started. Bound to: 127.13.75.190:44169
I20260923 15:06:07.261274 3215 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.75.190:44169 every 8 connection(s)
I20260923 15:06:07.261663 3147 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790175691491353-13614-0/minicluster-data/master-0/data/info.pb
I20260923 15:06:07.262639 3216 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:06:07.265086 3216 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 88742e54456d49f99af7b84a809b8678: Bootstrap starting.
I20260923 15:06:07.265647 3216 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 88742e54456d49f99af7b84a809b8678: Neither blocks nor log segments found. Creating new log.
I20260923 15:06:07.265901 3216 log.cc:824] T 00000000000000000000000000000000 P 88742e54456d49f99af7b84a809b8678: Log is configured to *not* fsync() on all Append() calls
I20260923 15:06:07.266609 3216 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 88742e54456d49f99af7b84a809b8678: No bootstrap required, opened a new log
I20260923 15:06:07.268286 3216 raft_consensus.cc:348] T 00000000000000000000000000000000 P 88742e54456d49f99af7b84a809b8678 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88742e54456d49f99af7b84a809b8678" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 44169 } }
I20260923 15:06:07.268437 3216 raft_consensus.cc:374] T 00000000000000000000000000000000 P 88742e54456d49f99af7b84a809b8678 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:06:07.268460 3216 raft_consensus.cc:729] T 00000000000000000000000000000000 P 88742e54456d49f99af7b84a809b8678 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 88742e54456d49f99af7b84a809b8678, State: Initialized, Role: FOLLOWER
I20260923 15:06:07.268600 3216 consensus_queue.cc:260] T 00000000000000000000000000000000 P 88742e54456d49f99af7b84a809b8678 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88742e54456d49f99af7b84a809b8678" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 44169 } }
I20260923 15:06:07.268693 3216 raft_consensus.cc:388] T 00000000000000000000000000000000 P 88742e54456d49f99af7b84a809b8678 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 15:06:07.268718 3216 raft_consensus.cc:482] T 00000000000000000000000000000000 P 88742e54456d49f99af7b84a809b8678 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 15:06:07.268749 3216 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 88742e54456d49f99af7b84a809b8678 [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:06:07.269608 3216 raft_consensus.cc:504] T 00000000000000000000000000000000 P 88742e54456d49f99af7b84a809b8678 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88742e54456d49f99af7b84a809b8678" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 44169 } }
I20260923 15:06:07.269709 3216 leader_election.cc:302] T 00000000000000000000000000000000 P 88742e54456d49f99af7b84a809b8678 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 88742e54456d49f99af7b84a809b8678; no voters:
I20260923 15:06:07.269852 3216 leader_election.cc:288] T 00000000000000000000000000000000 P 88742e54456d49f99af7b84a809b8678 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 15:06:07.269994 3219 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 88742e54456d49f99af7b84a809b8678 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 15:06:07.270110 3216 sys_catalog.cc:564] T 00000000000000000000000000000000 P 88742e54456d49f99af7b84a809b8678 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 15:06:07.270242 3219 raft_consensus.cc:686] T 00000000000000000000000000000000 P 88742e54456d49f99af7b84a809b8678 [term 1 LEADER]: Becoming Leader. State: Replica: 88742e54456d49f99af7b84a809b8678, State: Running, Role: LEADER
I20260923 15:06:07.270371 13614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKC4soy/build/release/bin/kudu as pid 3147
I20260923 15:06:07.270373 3219 consensus_queue.cc:237] T 00000000000000000000000000000000 P 88742e54456d49f99af7b84a809b8678 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88742e54456d49f99af7b84a809b8678" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 44169 } }
I20260923 15:06:07.270485 13614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_16.1790175691491353-13614-0/minicluster-data/master-0/wal/instance
I20260923 15:06:07.270663 13614 external_mini_cluster.cc:1428] Setting key e4e7b7135f8fbb60fee044e622952ab0
I20260923 15:06:07.270819 3220 sys_catalog.cc:455] T 00000000000000000000000000000000 P 88742e54456d49f99af7b84a809b8678 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "88742e54456d49f99af7b84a809b8678" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88742e54456d49f99af7b84a809b8678" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 44169 } } }
I20260923 15:06:07.270949 3220 sys_catalog.cc:458] T 00000000000000000000000000000000 P 88742e54456d49f99af7b84a809b8678 [sys.catalog]: This master's current role is: LEADER
I20260923 15:06:07.271123 3221 sys_catalog.cc:455] T 00000000000000000000000000000000 P 88742e54456d49f99af7b84a809b8678 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 88742e54456d49f99af7b84a809b8678. Latest consensus state: current_term: 1 leader_uuid: "88742e54456d49f99af7b84a809b8678" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "88742e54456d49f99af7b84a809b8678" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 44169 } } }
I20260923 15:06:07.271205 3221 sys_catalog.cc:458] T 00000000000000000000000000000000 P 88742e54456d49f99af7b84a809b8678 [sys.catalog]: This master's current role is: LEADER
I20260923 15:06:07.271657 3228 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 15:06:07.272539 3228 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 15:06:07.275142 3228 catalog_manager.cc:1347] Generated new cluster ID: 6c47fc4d4a764c2eb6e42dcab12c628f
I20260923 15:06:07.275221 3228 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 15:06:07.277208 3237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:07.271649 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:37407 (local address 127.13.75.190:44169)
0923 15:06:07.271921 (+ 272us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:07.271932 (+ 11us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:07.271969 (+ 37us) server_negotiation.cc:407] Connection header received
0923 15:06:07.272098 (+ 129us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:07.272106 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:07.272163 (+ 57us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:07.272320 (+ 157us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:07.272524 (+ 204us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:07.273592 (+ 1068us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:07.274749 (+ 1157us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:07.275492 (+ 743us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:07.275645 (+ 153us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:07.276478 (+ 833us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:06:07.276483 (+ 5us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:06:07.276483 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 15:06:07.276685 (+ 202us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:06:07.276731 (+ 46us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 15:06:07.276744 (+ 13us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:06:07.276828 (+ 84us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:07.276925 (+ 97us) server_negotiation.cc:299] Negotiation successful
0923 15:06:07.277000 (+ 75us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":208,"thread_start_us":71,"threads_started":1}
I20260923 15:06:07.278152 13614 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 15:06:07.286438 3237 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:07.283331 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:33105 (local address 127.13.75.190:44169)
0923 15:06:07.283401 (+ 70us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:07.283404 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:07.283549 (+ 145us) server_negotiation.cc:407] Connection header received
0923 15:06:07.283627 (+ 78us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:07.283628 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:07.283640 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:07.283697 (+ 57us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:07.283860 (+ 163us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:07.284368 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:07.285126 (+ 758us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:07.285380 (+ 254us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:07.285460 (+ 80us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:07.286154 (+ 694us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:06:07.286155 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:06:07.286156 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 15:06:07.286161 (+ 5us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:06:07.286175 (+ 14us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 15:06:07.286179 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:06:07.286251 (+ 72us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:07.286314 (+ 63us) server_negotiation.cc:299] Negotiation successful
0923 15:06:07.286344 (+ 30us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":41}
I20260923 15:06:07.287592 3228 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 15:06:07.288295 3228 catalog_manager.cc:1504] Loading token signing keys...
I20260923 15:06:07.298745 3228 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 88742e54456d49f99af7b84a809b8678: Generated new TSK 0
I20260923 15:06:07.299309 3228 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 15:06:07.309021 13614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKC4soy/build/release/bin/kudu with pid 3147
2026-09-23T15:06:07Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/16 (1408 ms)
[ RUN ] AuthTokenIssuingTest.ChannelConfidentiality/17
Loading random data
Initializing database '/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790175691491353-13614-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 15:06:07 dist-test-slave-cnb0 krb5kdc[3247](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 15:06:07 dist-test-slave-cnb0 krb5kdc[3247](info): set up 2 sockets
Sep 23 15:06:07 dist-test-slave-cnb0 krb5kdc[3247](info): commencing operation
krb5kdc: starting...
W20260923 15:06:08.368809 13614 mini_kdc.cc:121] Time spent starting KDC: real 1.042s user 0.001s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790175691491353-13614-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790175691491353-13614-0/krb5kdc/test-user.keytab.
Sep 23 15:06:08 dist-test-slave-cnb0 krb5kdc[3247](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175968, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T15:06:08Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T15:06:08Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.75.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.75.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.75.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.190.keytab.
Entry for principal kudu/127.13.75.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.190.keytab.
I20260923 15:06:08.490226 13614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKC4soy/build/release/bin/kudu
/tmp/dist-test-taskKC4soy/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790175691491353-13614-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790175691491353-13614-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790175691491353-13614-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790175691491353-13614-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.75.190:44389
--webserver_interface=127.13.75.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.190.keytab
--principal=kudu/127.13.75.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.75.148:46181
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.75.190:44389
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--user_acl=*
--rpc_authentication=disabled
--rpc_encryption=disabled
--rpc_encrypt_loopback_connections=false with env {KRB5CCNAME=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790175691491353-13614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790175691491353-13614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790175691491353-13614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:06:08.572054 3260 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:06:08.572269 3260 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:06:08.572293 3260 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:06:08.573379 3260 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 15:06:08.573419 3260 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:06:08.573434 3260 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:06:08.573447 3260 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 15:06:08.573460 3260 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 15:06:08.574966 3260 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.75.148:46181
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790175691491353-13614-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790175691491353-13614-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.75.190:44389
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.190.keytab
--principal=kudu/127.13.75.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.75.190:44389
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=disabled
--rpc_encryption=disabled
--superuser_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790175691491353-13614-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.75.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3260
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790175691491353-13614-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:06:08.575158 3260 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:06:08.575328 3260 file_cache.cc:493] Constructed file cache file cache with capacity 419430
I20260923 15:06:08.580180 3260 server_base.cc:1034] running on GCE node
W20260923 15:06:08.580118 3268 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:06:08.580098 3266 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:06:08.580091 3265 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:06:08.580617 3260 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:06:08.580926 3260 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:06:08.582091 3260 hybrid_clock.cc:648] HybridClock initialized: now 1790175968582065 us; error 41 us; skew 500 ppm
Sep 23 15:06:08 dist-test-slave-cnb0 krb5kdc[3247](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175968, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:06:08.584560 3260 init.cc:377] Logged in from keytab as kudu/127.13.75.190@KRBTEST.COM (short username kudu)
I20260923 15:06:08.584978 3260 webserver.cc:460] Webserver started at http://127.13.75.190:35933/ using document root <none> and password file <none>
I20260923 15:06:08.585227 3260 fs_manager.cc:359] Metadata directory not provided
I20260923 15:06:08.585295 3260 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:06:08.585433 3260 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:06:08.586463 3260 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790175691491353-13614-0/minicluster-data/master-0/data/instance:
uuid: "0794e01b38904007a66b40d5b487ba13"
format_stamp: "Formatted at 2026-09-23 15:06:08 on dist-test-slave-cnb0"
server_key: "262ff05cf0198ab1ee4b3ae40d13e988"
server_key_iv: "9aeb5ede4513c266c7ffd7db5b705bfe"
server_key_version: "encryptionkey@0"
I20260923 15:06:08.586818 3260 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790175691491353-13614-0/minicluster-data/master-0/wal/instance:
uuid: "0794e01b38904007a66b40d5b487ba13"
format_stamp: "Formatted at 2026-09-23 15:06:08 on dist-test-slave-cnb0"
server_key: "262ff05cf0198ab1ee4b3ae40d13e988"
server_key_iv: "9aeb5ede4513c266c7ffd7db5b705bfe"
server_key_version: "encryptionkey@0"
I20260923 15:06:08.588163 3260 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.001s
I20260923 15:06:08.588897 3275 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:06:08.589061 3260 fs_manager.cc:725] Time spent opening block manager: real 0.000s user 0.000s sys 0.001s
I20260923 15:06:08.589165 3260 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790175691491353-13614-0/minicluster-data/master-0/data,/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790175691491353-13614-0/minicluster-data/master-0/wal
uuid: "0794e01b38904007a66b40d5b487ba13"
format_stamp: "Formatted at 2026-09-23 15:06:08 on dist-test-slave-cnb0"
server_key: "262ff05cf0198ab1ee4b3ae40d13e988"
server_key_iv: "9aeb5ede4513c266c7ffd7db5b705bfe"
server_key_version: "encryptionkey@0"
I20260923 15:06:08.589236 3260 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790175691491353-13614-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790175691491353-13614-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790175691491353-13614-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:06:08.592550 3260 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:06:08.594339 3260 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:06:08.594487 3260 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:06:08.598438 3260 rpc_server.cc:307] RPC server started. Bound to: 127.13.75.190:44389
I20260923 15:06:08.598470 3327 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.75.190:44389 every 8 connection(s)
I20260923 15:06:08.598799 3260 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790175691491353-13614-0/minicluster-data/master-0/data/info.pb
I20260923 15:06:08.599786 3328 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:06:08.602044 3328 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0794e01b38904007a66b40d5b487ba13: Bootstrap starting.
I20260923 15:06:08.602627 3328 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 0794e01b38904007a66b40d5b487ba13: Neither blocks nor log segments found. Creating new log.
I20260923 15:06:08.602998 3328 log.cc:824] T 00000000000000000000000000000000 P 0794e01b38904007a66b40d5b487ba13: Log is configured to *not* fsync() on all Append() calls
I20260923 15:06:08.603662 3328 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 0794e01b38904007a66b40d5b487ba13: No bootstrap required, opened a new log
I20260923 15:06:08.604957 13614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKC4soy/build/release/bin/kudu as pid 3260
I20260923 15:06:08.605049 13614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.AuthTokenIssuingTest.ChannelConfidentiality_17.1790175691491353-13614-0/minicluster-data/master-0/wal/instance
I20260923 15:06:08.605284 13614 external_mini_cluster.cc:1428] Setting key 0c05da76da33a09bc46110ce2739c3a2
I20260923 15:06:08.605485 3328 raft_consensus.cc:348] T 00000000000000000000000000000000 P 0794e01b38904007a66b40d5b487ba13 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0794e01b38904007a66b40d5b487ba13" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 44389 } }
I20260923 15:06:08.605688 3328 raft_consensus.cc:374] T 00000000000000000000000000000000 P 0794e01b38904007a66b40d5b487ba13 [term 0 FOLLOWER]: Consensus starting up: Expiring failure detector timer to make a prompt election more likely
I20260923 15:06:08.605742 3328 raft_consensus.cc:729] T 00000000000000000000000000000000 P 0794e01b38904007a66b40d5b487ba13 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 0794e01b38904007a66b40d5b487ba13, State: Initialized, Role: FOLLOWER
I20260923 15:06:08.605890 3328 consensus_queue.cc:260] T 00000000000000000000000000000000 P 0794e01b38904007a66b40d5b487ba13 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0794e01b38904007a66b40d5b487ba13" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 44389 } }
I20260923 15:06:08.605997 3328 raft_consensus.cc:388] T 00000000000000000000000000000000 P 0794e01b38904007a66b40d5b487ba13 [term 0 FOLLOWER]: Only one voter in the Raft config. Triggering election immediately
I20260923 15:06:08.606046 3328 raft_consensus.cc:482] T 00000000000000000000000000000000 P 0794e01b38904007a66b40d5b487ba13 [term 0 FOLLOWER]: Starting leader election (initial election of a single-replica configuration)
I20260923 15:06:08.606102 3328 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 0794e01b38904007a66b40d5b487ba13 [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:06:08.607295 3328 raft_consensus.cc:504] T 00000000000000000000000000000000 P 0794e01b38904007a66b40d5b487ba13 [term 1 FOLLOWER]: Starting leader election with config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0794e01b38904007a66b40d5b487ba13" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 44389 } }
I20260923 15:06:08.607427 3328 leader_election.cc:302] T 00000000000000000000000000000000 P 0794e01b38904007a66b40d5b487ba13 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 1 responses out of 1 voters: 1 yes votes; 0 no votes. yes voters: 0794e01b38904007a66b40d5b487ba13; no voters:
I20260923 15:06:08.607512 3332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:08.605869 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:44393 (local address 127.13.75.190:44389)
0923 15:06:08.606081 (+ 212us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:08.606089 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:08.606124 (+ 35us) server_negotiation.cc:407] Connection header received
0923 15:06:08.606249 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:08.606254 (+ 5us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:08.606277 (+ 23us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:08.606390 (+ 113us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:08.607041 (+ 651us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:06:08.607044 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:06:08.607044 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 15:06:08.607048 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:06:08.607086 (+ 38us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 15:06:08.607097 (+ 11us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:06:08.607155 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:08.607258 (+ 103us) server_negotiation.cc:299] Negotiation successful
0923 15:06:08.607293 (+ 35us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":162,"thread_start_us":99,"threads_started":1}
I20260923 15:06:08.607585 3328 leader_election.cc:288] T 00000000000000000000000000000000 P 0794e01b38904007a66b40d5b487ba13 [CANDIDATE]: Term 1 election: Requested vote from peers
I20260923 15:06:08.607616 3333 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 0794e01b38904007a66b40d5b487ba13 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 15:06:08.607728 3333 raft_consensus.cc:686] T 00000000000000000000000000000000 P 0794e01b38904007a66b40d5b487ba13 [term 1 LEADER]: Becoming Leader. State: Replica: 0794e01b38904007a66b40d5b487ba13, State: Running, Role: LEADER
I20260923 15:06:08.607810 3333 consensus_queue.cc:237] T 00000000000000000000000000000000 P 0794e01b38904007a66b40d5b487ba13 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 1, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0794e01b38904007a66b40d5b487ba13" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 44389 } }
I20260923 15:06:08.608177 3328 sys_catalog.cc:564] T 00000000000000000000000000000000 P 0794e01b38904007a66b40d5b487ba13 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 15:06:08.608134 3333 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0794e01b38904007a66b40d5b487ba13 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 1 leader_uuid: "0794e01b38904007a66b40d5b487ba13" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0794e01b38904007a66b40d5b487ba13" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 44389 } } }
I20260923 15:06:08.608242 3333 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0794e01b38904007a66b40d5b487ba13 [sys.catalog]: This master's current role is: LEADER
I20260923 15:06:08.608378 3333 sys_catalog.cc:455] T 00000000000000000000000000000000 P 0794e01b38904007a66b40d5b487ba13 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 0794e01b38904007a66b40d5b487ba13. Latest consensus state: current_term: 1 leader_uuid: "0794e01b38904007a66b40d5b487ba13" committed_config { opid_index: -1 OBSOLETE_local: true peers { permanent_uuid: "0794e01b38904007a66b40d5b487ba13" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 44389 } } }
I20260923 15:06:08.608480 13614 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 15:06:08.608448 3333 sys_catalog.cc:458] T 00000000000000000000000000000000 P 0794e01b38904007a66b40d5b487ba13 [sys.catalog]: This master's current role is: LEADER
I20260923 15:06:08.608459 3337 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 15:06:08.608973 3337 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 15:06:08.611344 3332 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:08.609999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:41205 (local address 127.13.75.190:44389)
0923 15:06:08.610038 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:08.610040 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:08.610175 (+ 135us) server_negotiation.cc:407] Connection header received
0923 15:06:08.610251 (+ 76us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:08.610252 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:08.610267 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:08.610326 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:08.611009 (+ 683us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:06:08.611010 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:06:08.611010 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: PLAIN
0923 15:06:08.611014 (+ 4us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:06:08.611027 (+ 13us) server_negotiation.cc:1091] Received PLAIN auth, user=slave
0923 15:06:08.611031 (+ 4us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:06:08.611094 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:08.611266 (+ 172us) server_negotiation.cc:299] Negotiation successful
0923 15:06:08.611279 (+ 13us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260923 15:06:08.615707 3337 catalog_manager.cc:1347] Generated new cluster ID: ac617cdb107545e9a54a1914af3aa422
I20260923 15:06:08.615768 3337 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 15:06:08.623193 3337 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 15:06:08.623355 3337 catalog_manager.cc:1504] Loading token signing keys...
I20260923 15:06:08.629264 3337 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 0794e01b38904007a66b40d5b487ba13: Generated new TSK 0
I20260923 15:06:08.629485 3337 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 15:06:08.635336 13614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKC4soy/build/release/bin/kudu with pid 3260
2026-09-23T15:06:08Z chronyd exiting
[ OK ] AuthTokenIssuingTest.ChannelConfidentiality/17 (1326 ms)
[----------] 18 tests from AuthTokenIssuingTest (24631 ms total)
[----------] 2 tests from ConnectToFollowerMasterTest
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0
Loading random data
Initializing database '/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790175691491353-13614-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 15:06:08 dist-test-slave-cnb0 krb5kdc[3359](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 15:06:08 dist-test-slave-cnb0 krb5kdc[3359](info): set up 2 sockets
Sep 23 15:06:08 dist-test-slave-cnb0 krb5kdc[3359](info): commencing operation
krb5kdc: starting...
W20260923 15:06:09.688265 13614 mini_kdc.cc:121] Time spent starting KDC: real 1.035s user 0.000s sys 0.005s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790175691491353-13614-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790175691491353-13614-0/krb5kdc/test-user.keytab.
Sep 23 15:06:09 dist-test-slave-cnb0 krb5kdc[3359](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175969, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T15:06:09Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T15:06:09Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.75.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.75.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.75.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.190.keytab.
Entry for principal kudu/127.13.75.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.190.keytab.
I20260923 15:06:09.814329 13614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKC4soy/build/release/bin/kudu
/tmp/dist-test-taskKC4soy/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790175691491353-13614-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790175691491353-13614-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790175691491353-13614-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790175691491353-13614-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.75.190:40701
--webserver_interface=127.13.75.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.190.keytab
--principal=kudu/127.13.75.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.75.148:45473
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.75.190:40701,127.13.75.189:38455,127.13.75.188:38223
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790175691491353-13614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790175691491353-13614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790175691491353-13614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:06:09.898268 3372 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260923 15:06:09.898546 3372 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:06:09.898589 3372 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:06:09.898660 3372 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:06:09.899996 3372 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 15:06:09.900070 3372 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:06:09.900098 3372 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:06:09.900133 3372 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 15:06:09.900159 3372 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 15:06:09.901775 3372 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.75.148:45473
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790175691491353-13614-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790175691491353-13614-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.75.190:40701,127.13.75.189:38455,127.13.75.188:38223
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.190.keytab
--principal=kudu/127.13.75.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.75.190:40701
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790175691491353-13614-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.75.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3372
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790175691491353-13614-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:06:09.902076 3372 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:06:09.902321 3372 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:06:09.907493 3378 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:06:09.907478 3380 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:06:09.907478 3377 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:06:09.907532 3372 server_base.cc:1034] running on GCE node
I20260923 15:06:09.907948 3372 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:06:09.908267 3372 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:06:09.909420 3372 hybrid_clock.cc:648] HybridClock initialized: now 1790175969909403 us; error 31 us; skew 500 ppm
Sep 23 15:06:09 dist-test-slave-cnb0 krb5kdc[3359](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175969, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:06:09.912086 3372 init.cc:377] Logged in from keytab as kudu/127.13.75.190@KRBTEST.COM (short username kudu)
I20260923 15:06:09.912629 3372 webserver.cc:460] Webserver started at http://127.13.75.190:40913/ using document root <none> and password file <none>
I20260923 15:06:09.912858 3372 fs_manager.cc:359] Metadata directory not provided
I20260923 15:06:09.912925 3372 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:06:09.913071 3372 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:06:09.914160 3372 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790175691491353-13614-0/minicluster-data/master-0/data/instance:
uuid: "e9eb8aefa7294b548c2db794e5b64efd"
format_stamp: "Formatted at 2026-09-23 15:06:09 on dist-test-slave-cnb0"
server_key: "0768895146d2de1e0099ede2b00af556"
server_key_iv: "ce04fc5f8f1c7a812713d50dcb6e6a65"
server_key_version: "encryptionkey@0"
I20260923 15:06:09.914525 3372 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790175691491353-13614-0/minicluster-data/master-0/wal/instance:
uuid: "e9eb8aefa7294b548c2db794e5b64efd"
format_stamp: "Formatted at 2026-09-23 15:06:09 on dist-test-slave-cnb0"
server_key: "0768895146d2de1e0099ede2b00af556"
server_key_iv: "ce04fc5f8f1c7a812713d50dcb6e6a65"
server_key_version: "encryptionkey@0"
I20260923 15:06:09.916007 3372 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.002s sys 0.001s
I20260923 15:06:09.916858 3387 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:06:09.917106 3372 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:06:09.917255 3372 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790175691491353-13614-0/minicluster-data/master-0/data,/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790175691491353-13614-0/minicluster-data/master-0/wal
uuid: "e9eb8aefa7294b548c2db794e5b64efd"
format_stamp: "Formatted at 2026-09-23 15:06:09 on dist-test-slave-cnb0"
server_key: "0768895146d2de1e0099ede2b00af556"
server_key_iv: "ce04fc5f8f1c7a812713d50dcb6e6a65"
server_key_version: "encryptionkey@0"
I20260923 15:06:09.917336 3372 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790175691491353-13614-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790175691491353-13614-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790175691491353-13614-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:06:09.938817 3372 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:06:09.941151 3372 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:06:09.941327 3372 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:06:09.945653 3372 rpc_server.cc:307] RPC server started. Bound to: 127.13.75.190:40701
I20260923 15:06:09.945701 3439 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.75.190:40701 every 8 connection(s)
I20260923 15:06:09.946079 3372 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790175691491353-13614-0/minicluster-data/master-0/data/info.pb
I20260923 15:06:09.947118 3440 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:06:09.948689 3440 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.75.190" port: 40701 } has no permanent_uuid. Determining permanent_uuid...
I20260923 15:06:09.949750 13614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKC4soy/build/release/bin/kudu as pid 3372
I20260923 15:06:09.949864 13614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790175691491353-13614-0/minicluster-data/master-0/wal/instance
I20260923 15:06:09.950034 13614 external_mini_cluster.cc:1428] Setting key 2d42a37b6cf8f4342ab3c7c89a20df7c
Sep 23 15:06:09 dist-test-slave-cnb0 krb5kdc[3359](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175969, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.190@KRBTEST.COM for kudu/127.13.75.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.13.75.189@KRBTEST.COM; defaulting to no policy
I20260923 15:06:09.960759 3442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:09.949334 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:40701 (local address 127.0.0.1:39756)
0923 15:06:09.949502 (+ 168us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:09.949539 (+ 37us) client_negotiation.cc:174] Beginning negotiation
0923 15:06:09.950251 (+ 712us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:06:09.950550 (+ 299us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:06:09.950554 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:06:09.950986 (+ 432us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:06:09.951225 (+ 239us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:09.951231 (+ 6us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:09.952877 (+ 1646us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:09.952879 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:09.954029 (+ 1150us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:09.954032 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:09.954564 (+ 532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:09.954565 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:09.954657 (+ 92us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:09.955276 (+ 619us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:06:09.955287 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:06:09.957026 (+ 1739us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:06:09.959289 (+ 2263us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:06:09.959296 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:06:09.959298 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:06:09.959490 (+ 192us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:06:09.959731 (+ 241us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:06:09.959732 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:06:09.959732 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:06:09.959845 (+ 113us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:06:09.960238 (+ 393us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:06:09.960245 (+ 7us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:06:09.960309 (+ 64us) client_negotiation.cc:769] Sending connection context
0923 15:06:09.960401 (+ 92us) client_negotiation.cc:240] Negotiation successful
0923 15:06:09.960545 (+ 144us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":131,"thread_start_us":81,"threads_started":1}
I20260923 15:06:09.960765 3443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:09.949505 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39756 (local address 127.13.75.190:40701)
0923 15:06:09.949625 (+ 120us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:09.949630 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:09.949659 (+ 29us) server_negotiation.cc:407] Connection header received
0923 15:06:09.950406 (+ 747us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:09.950409 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:09.950426 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:09.950506 (+ 80us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:09.951342 (+ 836us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:09.952750 (+ 1408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:09.954109 (+ 1359us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:09.954450 (+ 341us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:09.954535 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:09.957137 (+ 2602us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:06:09.957143 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:06:09.957144 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:06:09.957151 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:06:09.959212 (+ 2061us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:09.959564 (+ 352us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:09.959567 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:09.959567 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:09.959648 (+ 81us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:09.959898 (+ 250us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:09.959899 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:09.959899 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:09.960152 (+ 253us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:06:09.960219 (+ 67us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:09.960470 (+ 251us) server_negotiation.cc:299] Negotiation successful
0923 15:06:09.960628 (+ 158us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":45,"server-negotiator.queue_time_us":85,"thread_start_us":59,"threads_started":1}
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.75.189@KRBTEST.COM" created.
I20260923 15:06:09.961720 3440 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.75.189" port: 38455 } has no permanent_uuid. Determining permanent_uuid...
I20260923 15:06:09.962147 3442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:09.962010 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.189:38455 (local address 127.0.0.1:40694)
0923 15:06:09.962063 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:09.962116 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.189:38455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260923 15:06:09.962316 3388 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.13.75.189:38455: connect: Connection refused (error 111)
I20260923 15:06:09.962750 3442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:09.962657 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.189:38455 (local address 127.0.0.1:40702)
0923 15:06:09.962701 (+ 44us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:09.962728 (+ 27us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.189:38455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260923 15:06:09.962857 3440 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.75.189:38455: Network error: Client connection negotiation failed: client connection to 127.13.75.189:38455: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.75.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.189.keytab.
Entry for principal kudu/127.13.75.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.189.keytab.
I20260923 15:06:09.979372 13614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKC4soy/build/release/bin/kudu
/tmp/dist-test-taskKC4soy/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790175691491353-13614-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790175691491353-13614-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790175691491353-13614-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790175691491353-13614-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.75.189:38455
--webserver_interface=127.13.75.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.189.keytab
--principal=kudu/127.13.75.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.75.148:45473
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.75.190:40701,127.13.75.189:38455,127.13.75.188:38223
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790175691491353-13614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790175691491353-13614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790175691491353-13614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260923 15:06:10.012120 3440 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.75.189" port: 38455 } attempt: 1
I20260923 15:06:10.012679 3442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:10.012514 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.189:38455 (local address 127.0.0.1:40704)
0923 15:06:10.012585 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:10.012638 (+ 53us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.189:38455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260923 15:06:10.013063 3442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:10.012970 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.189:38455 (local address 127.0.0.1:40714)
0923 15:06:10.013012 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:10.013042 (+ 30us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.189:38455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":27}
W20260923 15:06:10.013208 3440 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.75.189:38455: Network error: Client connection negotiation failed: client connection to 127.13.75.189:38455: connect: Connection refused (error 111)
W20260923 15:06:10.068248 3447 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260923 15:06:10.068473 3447 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:06:10.068491 3447 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:06:10.068513 3447 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:06:10.069641 3447 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 15:06:10.069679 3447 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:06:10.069692 3447 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:06:10.069705 3447 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 15:06:10.069716 3447 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 15:06:10.071261 3447 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.75.148:45473
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790175691491353-13614-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790175691491353-13614-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.13.75.190:40701,127.13.75.189:38455,127.13.75.188:38223
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.189.keytab
--principal=kudu/127.13.75.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.75.189:38455
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790175691491353-13614-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.13.75.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3447
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790175691491353-13614-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:06:10.071466 3447 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:06:10.071661 3447 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:06:10.077035 3452 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:06:10.077031 3453 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:06:10.077109 3455 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:06:10.077436 3447 server_base.cc:1034] running on GCE node
I20260923 15:06:10.077692 3447 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:06:10.077961 3447 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:06:10.079095 3447 hybrid_clock.cc:648] HybridClock initialized: now 1790175970079083 us; error 30 us; skew 500 ppm
Sep 23 15:06:10 dist-test-slave-cnb0 krb5kdc[3359](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175970, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:06:10.081403 3440 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.75.189" port: 38455 } attempt: 2
I20260923 15:06:10.081811 3442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:10.081664 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.189:38455 (local address 127.0.0.1:40716)
0923 15:06:10.081734 (+ 70us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:10.081776 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.189:38455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260923 15:06:10.081981 3447 init.cc:377] Logged in from keytab as kudu/127.13.75.189@KRBTEST.COM (short username kudu)
I20260923 15:06:10.082207 3442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:10.082064 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.189:38455 (local address 127.0.0.1:40726)
0923 15:06:10.082098 (+ 34us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:10.082180 (+ 82us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.189:38455: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":17}
W20260923 15:06:10.082304 3440 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.75.189:38455: Network error: Client connection negotiation failed: client connection to 127.13.75.189:38455: connect: Connection refused (error 111)
I20260923 15:06:10.082499 3447 webserver.cc:460] Webserver started at http://127.13.75.189:45137/ using document root <none> and password file <none>
I20260923 15:06:10.082756 3447 fs_manager.cc:359] Metadata directory not provided
I20260923 15:06:10.082823 3447 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:06:10.082991 3447 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:06:10.084057 3447 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790175691491353-13614-0/minicluster-data/master-1/data/instance:
uuid: "ea69d78e8d0a4f2ab829c0476e75435e"
format_stamp: "Formatted at 2026-09-23 15:06:10 on dist-test-slave-cnb0"
server_key: "d2750cef663b1b08349e1011a9cccd44"
server_key_iv: "2f9bb6b4b2ddb6ebc020a686a1f6d1b0"
server_key_version: "encryptionkey@0"
I20260923 15:06:10.084422 3447 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790175691491353-13614-0/minicluster-data/master-1/wal/instance:
uuid: "ea69d78e8d0a4f2ab829c0476e75435e"
format_stamp: "Formatted at 2026-09-23 15:06:10 on dist-test-slave-cnb0"
server_key: "d2750cef663b1b08349e1011a9cccd44"
server_key_iv: "2f9bb6b4b2ddb6ebc020a686a1f6d1b0"
server_key_version: "encryptionkey@0"
I20260923 15:06:10.085831 3447 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 15:06:10.086679 3462 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:06:10.086946 3447 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:06:10.087128 3447 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790175691491353-13614-0/minicluster-data/master-1/data,/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790175691491353-13614-0/minicluster-data/master-1/wal
uuid: "ea69d78e8d0a4f2ab829c0476e75435e"
format_stamp: "Formatted at 2026-09-23 15:06:10 on dist-test-slave-cnb0"
server_key: "d2750cef663b1b08349e1011a9cccd44"
server_key_iv: "2f9bb6b4b2ddb6ebc020a686a1f6d1b0"
server_key_version: "encryptionkey@0"
I20260923 15:06:10.087224 3447 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790175691491353-13614-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790175691491353-13614-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790175691491353-13614-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:06:10.103117 3447 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:06:10.105515 3447 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:06:10.105695 3447 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:06:10.109899 3447 rpc_server.cc:307] RPC server started. Bound to: 127.13.75.189:38455
I20260923 15:06:10.109948 3514 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.75.189:38455 every 8 connection(s)
I20260923 15:06:10.110368 3447 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790175691491353-13614-0/minicluster-data/master-1/data/info.pb
I20260923 15:06:10.111420 3515 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:06:10.112900 3515 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.75.190" port: 40701 } has no permanent_uuid. Determining permanent_uuid...
I20260923 15:06:10.114960 13614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKC4soy/build/release/bin/kudu as pid 3447
I20260923 15:06:10.115082 13614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790175691491353-13614-0/minicluster-data/master-1/wal/instance
I20260923 15:06:10.115283 13614 external_mini_cluster.cc:1428] Setting key f85f26c54c1131221eb43a3b83e6e76e
Sep 23 15:06:10 dist-test-slave-cnb0 krb5kdc[3359](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175970, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.189@KRBTEST.COM for kudu/127.13.75.190@KRBTEST.COM
WARNING: no policy specified for kudu/127.13.75.188@KRBTEST.COM; defaulting to no policy
I20260923 15:06:10.124045 3443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:10.113636 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39772 (local address 127.13.75.190:40701)
0923 15:06:10.113735 (+ 99us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:10.113737 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:10.113923 (+ 186us) server_negotiation.cc:407] Connection header received
0923 15:06:10.114445 (+ 522us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:10.114447 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:10.114460 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:10.114516 (+ 56us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:10.115636 (+ 1120us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:10.116224 (+ 588us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:10.117516 (+ 1292us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:10.117768 (+ 252us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:10.117830 (+ 62us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:10.120446 (+ 2616us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:06:10.120448 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:06:10.120449 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:06:10.120458 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:06:10.122293 (+ 1835us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:10.122737 (+ 444us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:10.122738 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:10.122739 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:10.122804 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:10.123136 (+ 332us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:10.123138 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:10.123138 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:10.123303 (+ 165us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:06:10.123390 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:10.123833 (+ 443us) server_negotiation.cc:299] Negotiation successful
0923 15:06:10.123933 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":69}
I20260923 15:06:10.124105 3517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:10.113593 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:40701 (local address 127.0.0.1:39772)
0923 15:06:10.113799 (+ 206us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:10.113838 (+ 39us) client_negotiation.cc:174] Beginning negotiation
0923 15:06:10.114356 (+ 518us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:06:10.114626 (+ 270us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:06:10.114630 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:06:10.115154 (+ 524us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:06:10.115509 (+ 355us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:10.115517 (+ 8us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:10.116341 (+ 824us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:10.116343 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:10.117412 (+ 1069us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:10.117415 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:10.117865 (+ 450us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:10.117866 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:10.117995 (+ 129us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:10.118597 (+ 602us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:06:10.118608 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:06:10.120311 (+ 1703us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:06:10.122390 (+ 2079us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:06:10.122399 (+ 9us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:06:10.122401 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:06:10.122646 (+ 245us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:06:10.122925 (+ 279us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:06:10.122926 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:06:10.122926 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:06:10.123063 (+ 137us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:06:10.123415 (+ 352us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:06:10.123423 (+ 8us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:06:10.123587 (+ 164us) client_negotiation.cc:769] Sending connection context
0923 15:06:10.123725 (+ 138us) client_negotiation.cc:240] Negotiation successful
0923 15:06:10.123922 (+ 197us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":165,"thread_start_us":104,"threads_started":1}
I20260923 15:06:10.124974 3515 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.75.189" port: 38455 } has no permanent_uuid. Determining permanent_uuid...
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.75.188@KRBTEST.COM" created.
Sep 23 15:06:10 dist-test-slave-cnb0 krb5kdc[3359](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175970, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.189@KRBTEST.COM for kudu/127.13.75.189@KRBTEST.COM
I20260923 15:06:10.133705 3517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:10.125272 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.189:38455 (local address 127.0.0.1:40738)
0923 15:06:10.125324 (+ 52us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:10.125335 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 15:06:10.125378 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:06:10.125700 (+ 322us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:06:10.125702 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:06:10.125844 (+ 142us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:06:10.125939 (+ 95us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:10.125941 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:10.126986 (+ 1045us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:10.126987 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:10.127485 (+ 498us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:10.127486 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:10.127988 (+ 502us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:10.127989 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:10.128054 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:10.128577 (+ 523us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:06:10.128583 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:06:10.130007 (+ 1424us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:06:10.132428 (+ 2421us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:06:10.132430 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:06:10.132430 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:06:10.132663 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:06:10.132937 (+ 274us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:06:10.132938 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:06:10.132939 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:06:10.133010 (+ 71us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:06:10.133431 (+ 421us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:06:10.133432 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:06:10.133475 (+ 43us) client_negotiation.cc:769] Sending connection context
0923 15:06:10.133548 (+ 73us) client_negotiation.cc:240] Negotiation successful
0923 15:06:10.133631 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":31}
I20260923 15:06:10.133838 3519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:10.125365 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40738 (local address 127.13.75.189:38455)
0923 15:06:10.125528 (+ 163us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:10.125532 (+ 4us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:10.125550 (+ 18us) server_negotiation.cc:407] Connection header received
0923 15:06:10.125566 (+ 16us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:10.125568 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:10.125578 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:10.125636 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:10.126017 (+ 381us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:10.126846 (+ 829us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:10.127574 (+ 728us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:10.127900 (+ 326us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:10.127989 (+ 89us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:10.130093 (+ 2104us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:06:10.130103 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:06:10.130103 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:06:10.130111 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:06:10.132311 (+ 2200us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:10.132760 (+ 449us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:10.132764 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:10.132765 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:10.132835 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:10.133120 (+ 285us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:10.133121 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:10.133121 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:10.133341 (+ 220us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:06:10.133436 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:10.133611 (+ 175us) server_negotiation.cc:299] Negotiation successful
0923 15:06:10.133711 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":137,"thread_start_us":94,"threads_started":1}
I20260923 15:06:10.134829 3515 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.75.188" port: 38223 } has no permanent_uuid. Determining permanent_uuid...
I20260923 15:06:10.135279 3517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:10.135129 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.188:38223 (local address 127.0.0.1:48574)
0923 15:06:10.135186 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:10.135245 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.188:38223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":31}
W20260923 15:06:10.135448 3466 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.13.75.188:38223: connect: Connection refused (error 111)
I20260923 15:06:10.135901 3517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:10.135800 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.188:38223 (local address 127.0.0.1:48584)
0923 15:06:10.135853 (+ 53us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:10.135881 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.188:38223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":39}
W20260923 15:06:10.136029 3515 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.75.188:38223: Network error: Client connection negotiation failed: client connection to 127.13.75.188:38223: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.75.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.188.keytab.
Entry for principal kudu/127.13.75.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.188.keytab.
I20260923 15:06:10.146711 13614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKC4soy/build/release/bin/kudu
/tmp/dist-test-taskKC4soy/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790175691491353-13614-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790175691491353-13614-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790175691491353-13614-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790175691491353-13614-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.75.188:38223
--webserver_interface=127.13.75.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.188.keytab
--principal=kudu/127.13.75.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.75.148:45473
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.75.190:40701,127.13.75.189:38455,127.13.75.188:38223
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=optional with env {KRB5CCNAME=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790175691491353-13614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790175691491353-13614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790175691491353-13614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260923 15:06:10.173493 3440 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.75.189" port: 38455 } attempt: 3
Sep 23 15:06:10 dist-test-slave-cnb0 krb5kdc[3359](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175969, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.190@KRBTEST.COM for kudu/127.13.75.189@KRBTEST.COM
I20260923 15:06:10.180861 3519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:10.174005 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40746 (local address 127.13.75.189:38455)
0923 15:06:10.174076 (+ 71us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:10.174079 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:10.174108 (+ 29us) server_negotiation.cc:407] Connection header received
0923 15:06:10.174138 (+ 30us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:10.174139 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:10.174154 (+ 15us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:10.174219 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:10.174568 (+ 349us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:10.175133 (+ 565us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:10.175890 (+ 757us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:10.176114 (+ 224us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:10.176206 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:10.178198 (+ 1992us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:06:10.178200 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:06:10.178201 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:06:10.178215 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:06:10.179669 (+ 1454us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:10.180037 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:10.180038 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:10.180039 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:10.180102 (+ 63us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:10.180303 (+ 201us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:10.180303 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:10.180304 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:10.180416 (+ 112us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:06:10.180518 (+ 102us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:10.180725 (+ 207us) server_negotiation.cc:299] Negotiation successful
0923 15:06:10.180792 (+ 67us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":44}
I20260923 15:06:10.180902 3442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:10.173927 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.189:38455 (local address 127.0.0.1:40746)
0923 15:06:10.173986 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:10.174003 (+ 17us) client_negotiation.cc:174] Beginning negotiation
0923 15:06:10.174070 (+ 67us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:06:10.174227 (+ 157us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:06:10.174228 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:06:10.174414 (+ 186us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:06:10.174479 (+ 65us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:10.174480 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:10.175269 (+ 789us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:10.175270 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:10.175802 (+ 532us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:10.175804 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:10.176215 (+ 411us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:10.176216 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:10.176298 (+ 82us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:10.176805 (+ 507us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:06:10.176813 (+ 8us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:06:10.178103 (+ 1290us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:06:10.179781 (+ 1678us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:06:10.179782 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:06:10.179783 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:06:10.179965 (+ 182us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:06:10.180168 (+ 203us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:06:10.180169 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:06:10.180170 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:06:10.180232 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:06:10.180540 (+ 308us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:06:10.180541 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:06:10.180606 (+ 65us) client_negotiation.cc:769] Sending connection context
0923 15:06:10.180687 (+ 81us) client_negotiation.cc:240] Negotiation successful
0923 15:06:10.180796 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":34}
I20260923 15:06:10.181377 3440 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.75.188" port: 38223 } has no permanent_uuid. Determining permanent_uuid...
I20260923 15:06:10.181728 3442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:10.181614 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.188:38223 (local address 127.0.0.1:48596)
0923 15:06:10.181669 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:10.181705 (+ 36us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.188:38223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":37}
I20260923 15:06:10.182052 3442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:10.181944 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.188:38223 (local address 127.0.0.1:48610)
0923 15:06:10.182002 (+ 58us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:10.182028 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.188:38223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":43}
W20260923 15:06:10.182145 3440 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.75.188:38223: Network error: Client connection negotiation failed: client connection to 127.13.75.188:38223: connect: Connection refused (error 111)
I20260923 15:06:10.185236 3515 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.75.188" port: 38223 } attempt: 1
I20260923 15:06:10.185678 3517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:10.185559 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.188:38223 (local address 127.0.0.1:48626)
0923 15:06:10.185616 (+ 57us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:10.185653 (+ 37us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.188:38223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":38}
I20260923 15:06:10.186025 3517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:10.185935 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.188:38223 (local address 127.0.0.1:48634)
0923 15:06:10.185975 (+ 40us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:10.186004 (+ 29us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.188:38223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":25}
W20260923 15:06:10.186111 3515 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.75.188:38223: Network error: Client connection negotiation failed: client connection to 127.13.75.188:38223: connect: Connection refused (error 111)
I20260923 15:06:10.213318 3440 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.75.188" port: 38223 } attempt: 1
I20260923 15:06:10.213930 3442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:10.213754 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.188:38223 (local address 127.0.0.1:48638)
0923 15:06:10.213831 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:10.213887 (+ 56us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.188:38223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":47}
I20260923 15:06:10.214332 3442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:10.214245 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.188:38223 (local address 127.0.0.1:48654)
0923 15:06:10.214288 (+ 43us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:10.214314 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.188:38223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":28}
W20260923 15:06:10.214457 3440 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.75.188:38223: Network error: Client connection negotiation failed: client connection to 127.13.75.188:38223: connect: Connection refused (error 111)
W20260923 15:06:10.232757 3522 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260923 15:06:10.232975 3522 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:06:10.232993 3522 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:06:10.233016 3522 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:06:10.234180 3522 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 15:06:10.234227 3522 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:06:10.234243 3522 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:06:10.234256 3522 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 15:06:10.234269 3522 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 15:06:10.235867 3522 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.75.148:45473
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790175691491353-13614-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790175691491353-13614-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.13.75.190:40701,127.13.75.189:38455,127.13.75.188:38223
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.188.keytab
--principal=kudu/127.13.75.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.75.188:38223
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790175691491353-13614-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.13.75.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3522
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790175691491353-13614-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:06:10.236123 3522 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:06:10.236356 3522 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:06:10.241596 3530 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:06:10.241596 3527 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:06:10.241735 3522 server_base.cc:1034] running on GCE node
W20260923 15:06:10.241575 3528 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:06:10.242182 3522 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:06:10.242502 3522 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:06:10.243678 3522 hybrid_clock.cc:648] HybridClock initialized: now 1790175970243672 us; error 51 us; skew 500 ppm
Sep 23 15:06:10 dist-test-slave-cnb0 krb5kdc[3359](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175970, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:06:10.246268 3522 init.cc:377] Logged in from keytab as kudu/127.13.75.188@KRBTEST.COM (short username kudu)
I20260923 15:06:10.246728 3522 webserver.cc:460] Webserver started at http://127.13.75.188:43533/ using document root <none> and password file <none>
I20260923 15:06:10.246968 3522 fs_manager.cc:359] Metadata directory not provided
I20260923 15:06:10.247016 3522 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:06:10.247123 3522 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:06:10.248152 3522 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790175691491353-13614-0/minicluster-data/master-2/data/instance:
uuid: "6bd07faf001a401bb1f535360d89de93"
format_stamp: "Formatted at 2026-09-23 15:06:10 on dist-test-slave-cnb0"
server_key: "bec9b3dc4f061d4ac27297e4b68ceae6"
server_key_iv: "eb15a8cc74c658c94144873856342e58"
server_key_version: "encryptionkey@0"
I20260923 15:06:10.248452 3522 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790175691491353-13614-0/minicluster-data/master-2/wal/instance:
uuid: "6bd07faf001a401bb1f535360d89de93"
format_stamp: "Formatted at 2026-09-23 15:06:10 on dist-test-slave-cnb0"
server_key: "bec9b3dc4f061d4ac27297e4b68ceae6"
server_key_iv: "eb15a8cc74c658c94144873856342e58"
server_key_version: "encryptionkey@0"
I20260923 15:06:10.249758 3522 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 15:06:10.250582 3537 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:06:10.250833 3522 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:06:10.250975 3522 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790175691491353-13614-0/minicluster-data/master-2/data,/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790175691491353-13614-0/minicluster-data/master-2/wal
uuid: "6bd07faf001a401bb1f535360d89de93"
format_stamp: "Formatted at 2026-09-23 15:06:10 on dist-test-slave-cnb0"
server_key: "bec9b3dc4f061d4ac27297e4b68ceae6"
server_key_iv: "eb15a8cc74c658c94144873856342e58"
server_key_version: "encryptionkey@0"
I20260923 15:06:10.251065 3522 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790175691491353-13614-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790175691491353-13614-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790175691491353-13614-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:06:10.254326 3515 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.75.188" port: 38223 } attempt: 2
I20260923 15:06:10.254787 3517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:10.254649 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.188:38223 (local address 127.0.0.1:48660)
0923 15:06:10.254708 (+ 59us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:10.254750 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.188:38223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":33}
I20260923 15:06:10.255260 3517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:10.255126 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.188:38223 (local address 127.0.0.1:48666)
0923 15:06:10.255203 (+ 77us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:10.255237 (+ 34us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.188:38223: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
W20260923 15:06:10.255434 3515 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.75.188:38223: Network error: Client connection negotiation failed: client connection to 127.13.75.188:38223: connect: Connection refused (error 111)
I20260923 15:06:10.264339 3522 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:06:10.266595 3522 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:06:10.266782 3522 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:06:10.271360 3522 rpc_server.cc:307] RPC server started. Bound to: 127.13.75.188:38223
I20260923 15:06:10.271390 3589 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.75.188:38223 every 8 connection(s)
I20260923 15:06:10.271799 3522 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790175691491353-13614-0/minicluster-data/master-2/data/info.pb
I20260923 15:06:10.271955 13614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKC4soy/build/release/bin/kudu as pid 3522
I20260923 15:06:10.272074 13614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_0.1790175691491353-13614-0/minicluster-data/master-2/wal/instance
I20260923 15:06:10.272279 13614 external_mini_cluster.cc:1428] Setting key 94e399f6652c3760e858bdce9ca6c0cc
I20260923 15:06:10.273237 3590 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:06:10.275295 3590 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.75.190" port: 40701 } has no permanent_uuid. Determining permanent_uuid...
Sep 23 15:06:10 dist-test-slave-cnb0 krb5kdc[3359](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175969, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.75.188@KRBTEST.COM
Sep 23 15:06:10 dist-test-slave-cnb0 krb5kdc[3359](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175970, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.188@KRBTEST.COM for kudu/127.13.75.190@KRBTEST.COM
I20260923 15:06:10.283831 3593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:10.272862 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48676 (local address 127.13.75.188:38223)
0923 15:06:10.273104 (+ 242us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:10.273112 (+ 8us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:10.273139 (+ 27us) server_negotiation.cc:407] Connection header received
0923 15:06:10.273624 (+ 485us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:10.273632 (+ 8us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:10.273682 (+ 50us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:10.273805 (+ 123us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:10.274612 (+ 807us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:10.275645 (+ 1033us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:10.277019 (+ 1374us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:10.277695 (+ 676us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:10.277840 (+ 145us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:10.280456 (+ 2616us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:06:10.280462 (+ 6us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:06:10.280462 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:06:10.280470 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:06:10.282115 (+ 1645us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:10.282491 (+ 376us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:10.282494 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:10.282495 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:10.282612 (+ 117us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:10.282824 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:10.282825 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:10.282826 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:10.283187 (+ 361us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:06:10.283337 (+ 150us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:10.283442 (+ 105us) server_negotiation.cc:299] Negotiation successful
0923 15:06:10.283587 (+ 145us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":151,"thread_start_us":91,"threads_started":1}
I20260923 15:06:10.284898 3443 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:10.276098 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39776 (local address 127.13.75.190:40701)
0923 15:06:10.276258 (+ 160us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:10.276260 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:10.276507 (+ 247us) server_negotiation.cc:407] Connection header received
0923 15:06:10.276633 (+ 126us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:10.276635 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:10.276645 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:10.276705 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:10.277300 (+ 595us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:10.278049 (+ 749us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:10.278966 (+ 917us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:10.279194 (+ 228us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:10.279293 (+ 99us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:10.281869 (+ 2576us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:06:10.281871 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:06:10.281871 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:06:10.281880 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:06:10.283592 (+ 1712us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:10.284004 (+ 412us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:10.284005 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:10.284005 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:10.284067 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:10.284275 (+ 208us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:10.284276 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:10.284276 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:10.284414 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:06:10.284503 (+ 89us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:10.284709 (+ 206us) server_negotiation.cc:299] Negotiation successful
0923 15:06:10.284802 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":89}
I20260923 15:06:10.284929 3594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:10.275991 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:40701 (local address 127.0.0.1:39776)
0923 15:06:10.276370 (+ 379us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:10.276422 (+ 52us) client_negotiation.cc:174] Beginning negotiation
0923 15:06:10.276511 (+ 89us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:06:10.276752 (+ 241us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:06:10.276755 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:06:10.277125 (+ 370us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:06:10.277196 (+ 71us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:10.277198 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:10.278154 (+ 956us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:10.278155 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:10.278813 (+ 658us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:10.278815 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:10.279332 (+ 517us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:10.279333 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:10.279413 (+ 80us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:10.279986 (+ 573us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:06:10.279995 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:06:10.281768 (+ 1773us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:06:10.283681 (+ 1913us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:06:10.283688 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:06:10.283690 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:06:10.283918 (+ 228us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:06:10.284139 (+ 221us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:06:10.284140 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:06:10.284140 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:06:10.284212 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:06:10.284493 (+ 281us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:06:10.284495 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:06:10.284557 (+ 62us) client_negotiation.cc:769] Sending connection context
0923 15:06:10.284674 (+ 117us) client_negotiation.cc:240] Negotiation successful
0923 15:06:10.284807 (+ 133us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":170,"thread_start_us":114,"threads_started":1}
I20260923 15:06:10.285405 3590 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.75.189" port: 38455 } has no permanent_uuid. Determining permanent_uuid...
Sep 23 15:06:10 dist-test-slave-cnb0 krb5kdc[3359](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175970, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.188@KRBTEST.COM for kudu/127.13.75.189@KRBTEST.COM
Sep 23 15:06:10 dist-test-slave-cnb0 krb5kdc[3359](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175969, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.75.190@KRBTEST.COM
I20260923 15:06:10.289713 3440 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.75.188" port: 38223 } attempt: 2
I20260923 15:06:10.293391 3519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:10.285853 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40752 (local address 127.13.75.189:38455)
0923 15:06:10.285894 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:10.285896 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:10.285906 (+ 10us) server_negotiation.cc:407] Connection header received
0923 15:06:10.285932 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:10.285933 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:10.285947 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:10.285988 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:10.286262 (+ 274us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:10.286706 (+ 444us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:10.287443 (+ 737us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:10.287673 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:10.287757 (+ 84us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:10.289510 (+ 1753us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:06:10.289512 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:06:10.289512 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:06:10.289520 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:06:10.291363 (+ 1843us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:10.291837 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:10.291838 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:10.291838 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:10.291910 (+ 72us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:10.292181 (+ 271us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:10.292182 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:10.292183 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:10.292597 (+ 414us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:06:10.292723 (+ 126us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:10.293221 (+ 498us) server_negotiation.cc:299] Negotiation successful
0923 15:06:10.293298 (+ 77us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":18}
I20260923 15:06:10.293391 3594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:10.285783 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.189:38455 (local address 127.0.0.1:40752)
0923 15:06:10.285838 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:10.285849 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 15:06:10.285888 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:06:10.286000 (+ 112us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:06:10.286001 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:06:10.286143 (+ 142us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:06:10.286197 (+ 54us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:10.286198 (+ 1us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:10.286789 (+ 591us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:10.286790 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:10.287358 (+ 568us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:10.287360 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:10.287759 (+ 399us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:10.287760 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:10.287832 (+ 72us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:10.288193 (+ 361us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:06:10.288197 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:06:10.289416 (+ 1219us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:06:10.291455 (+ 2039us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:06:10.291456 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:06:10.291457 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:06:10.291756 (+ 299us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:06:10.292004 (+ 248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:06:10.292006 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:06:10.292006 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:06:10.292074 (+ 68us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:06:10.293030 (+ 956us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:06:10.293031 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:06:10.293083 (+ 52us) client_negotiation.cc:769] Sending connection context
0923 15:06:10.293185 (+ 102us) client_negotiation.cc:240] Negotiation successful
0923 15:06:10.293290 (+ 105us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":36}
I20260923 15:06:10.294139 3590 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.75.188" port: 38223 } has no permanent_uuid. Determining permanent_uuid...
I20260923 15:06:10.294101 3595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:10.284716 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39778 (local address 127.13.75.190:40701)
0923 15:06:10.284905 (+ 189us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:10.284907 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:10.284914 (+ 7us) server_negotiation.cc:407] Connection header received
0923 15:06:10.284936 (+ 22us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:10.284937 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:10.284948 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:10.285041 (+ 93us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:10.285658 (+ 617us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:10.286385 (+ 727us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:10.287121 (+ 736us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:10.287346 (+ 225us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:10.287414 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:10.290514 (+ 3100us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:06:10.290515 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:06:10.290516 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:06:10.290523 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:06:10.292668 (+ 2145us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:10.293012 (+ 344us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:10.293013 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:10.293013 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:10.293075 (+ 62us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:10.293371 (+ 296us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:10.293372 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:10.293372 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:10.293510 (+ 138us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:06:10.293819 (+ 309us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:10.293870 (+ 51us) server_negotiation.cc:299] Negotiation successful
0923 15:06:10.293958 (+ 88us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":165,"thread_start_us":111,"threads_started":1}
Sep 23 15:06:10 dist-test-slave-cnb0 krb5kdc[3359](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175969, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.190@KRBTEST.COM for kudu/127.13.75.188@KRBTEST.COM
Sep 23 15:06:10 dist-test-slave-cnb0 krb5kdc[3359](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175970, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.188@KRBTEST.COM for kudu/127.13.75.188@KRBTEST.COM
Sep 23 15:06:10 dist-test-slave-cnb0 krb5kdc[3359](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175969, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.75.189@KRBTEST.COM
I20260923 15:06:10.300453 3442 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:10.290026 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.188:38223 (local address 127.0.0.1:48684)
0923 15:06:10.290377 (+ 351us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:10.290390 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0923 15:06:10.290448 (+ 58us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:06:10.290710 (+ 262us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:06:10.290712 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:06:10.291353 (+ 641us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:06:10.291402 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:10.291404 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:10.292346 (+ 942us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:10.292347 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:10.293285 (+ 938us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:10.293288 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:10.294679 (+ 1391us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:10.294680 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:10.294758 (+ 78us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:10.295299 (+ 541us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:06:10.295305 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:06:10.296934 (+ 1629us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:06:10.299304 (+ 2370us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:06:10.299306 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:06:10.299307 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:06:10.299538 (+ 231us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:06:10.299773 (+ 235us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:06:10.299773 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:06:10.299774 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:06:10.299836 (+ 62us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:06:10.300124 (+ 288us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:06:10.300125 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:06:10.300179 (+ 54us) client_negotiation.cc:769] Sending connection context
0923 15:06:10.300259 (+ 80us) client_negotiation.cc:240] Negotiation successful
0923 15:06:10.300368 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":328}
I20260923 15:06:10.300858 3593 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:10.290084 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48684 (local address 127.13.75.188:38223)
0923 15:06:10.290171 (+ 87us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:10.290173 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:10.290466 (+ 293us) server_negotiation.cc:407] Connection header received
0923 15:06:10.290499 (+ 33us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:10.290500 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:10.290513 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:10.290697 (+ 184us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:10.291505 (+ 808us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:10.292245 (+ 740us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:10.293371 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:10.294616 (+ 1245us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:10.294683 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:10.297027 (+ 2344us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:06:10.297028 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:06:10.297028 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:06:10.297036 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:06:10.299188 (+ 2152us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:10.299639 (+ 451us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:10.299640 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:10.299640 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:10.299696 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:10.299909 (+ 213us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:10.299910 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:10.299910 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:10.300039 (+ 129us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:06:10.300108 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:10.300672 (+ 564us) server_negotiation.cc:299] Negotiation successful
0923 15:06:10.300768 (+ 96us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260923 15:06:10.303077 3594 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:10.294430 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.188:38223 (local address 127.0.0.1:48690)
0923 15:06:10.294731 (+ 301us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:10.294741 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 15:06:10.294774 (+ 33us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:06:10.295173 (+ 399us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:06:10.295174 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:06:10.295354 (+ 180us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:06:10.295396 (+ 42us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:10.295398 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:10.296634 (+ 1236us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:10.296635 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:10.297380 (+ 745us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:10.297383 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:10.297787 (+ 404us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:10.297788 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:10.297838 (+ 50us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:10.298228 (+ 390us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:06:10.298234 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:06:10.299897 (+ 1663us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:06:10.302010 (+ 2113us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:06:10.302011 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:06:10.302011 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:06:10.302234 (+ 223us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:06:10.302416 (+ 182us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:06:10.302417 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:06:10.302417 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:06:10.302460 (+ 43us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:06:10.302757 (+ 297us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:06:10.302758 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:06:10.302807 (+ 49us) client_negotiation.cc:769] Sending connection context
0923 15:06:10.302866 (+ 59us) client_negotiation.cc:240] Negotiation successful
0923 15:06:10.302987 (+ 121us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":287}
I20260923 15:06:10.303313 3440 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e9eb8aefa7294b548c2db794e5b64efd: Bootstrap starting.
I20260923 15:06:10.303445 3596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:10.294450 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48690 (local address 127.13.75.188:38223)
0923 15:06:10.294817 (+ 367us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:10.294819 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:10.294829 (+ 10us) server_negotiation.cc:407] Connection header received
0923 15:06:10.294850 (+ 21us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:10.294851 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:10.294865 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:10.294976 (+ 111us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:10.295490 (+ 514us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:10.296546 (+ 1056us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:10.297471 (+ 925us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:10.297702 (+ 231us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:10.297788 (+ 86us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:10.299974 (+ 2186us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:06:10.299975 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:06:10.299975 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:06:10.299981 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:06:10.301928 (+ 1947us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:10.302310 (+ 382us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:10.302311 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:10.302311 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:10.302360 (+ 49us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:10.302514 (+ 154us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:10.302515 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:10.302515 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:10.302671 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:06:10.302735 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:10.303256 (+ 521us) server_negotiation.cc:299] Negotiation successful
0923 15:06:10.303335 (+ 79us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":330,"thread_start_us":115,"threads_started":1}
I20260923 15:06:10.303761 3519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:10.295922 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40766 (local address 127.13.75.189:38455)
0923 15:06:10.295974 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:10.295977 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:10.295987 (+ 10us) server_negotiation.cc:407] Connection header received
0923 15:06:10.296013 (+ 26us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:10.296014 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:10.296027 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:10.296087 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:10.296752 (+ 665us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:10.297268 (+ 516us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:10.298039 (+ 771us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:10.298342 (+ 303us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:10.298405 (+ 63us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:10.300836 (+ 2431us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:06:10.300837 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:06:10.300838 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:06:10.300846 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:06:10.302369 (+ 1523us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:10.302742 (+ 373us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:10.302744 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:10.302744 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:10.302801 (+ 57us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:10.303035 (+ 234us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:10.303036 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:10.303036 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:10.303184 (+ 148us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:06:10.303356 (+ 172us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:10.303591 (+ 235us) server_negotiation.cc:299] Negotiation successful
0923 15:06:10.303669 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":28}
I20260923 15:06:10.304108 3440 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P e9eb8aefa7294b548c2db794e5b64efd: Neither blocks nor log segments found. Creating new log.
I20260923 15:06:10.304417 3440 log.cc:824] T 00000000000000000000000000000000 P e9eb8aefa7294b548c2db794e5b64efd: Log is configured to *not* fsync() on all Append() calls
I20260923 15:06:10.304924 3590 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6bd07faf001a401bb1f535360d89de93: Bootstrap starting.
I20260923 15:06:10.305292 3440 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P e9eb8aefa7294b548c2db794e5b64efd: No bootstrap required, opened a new log
I20260923 15:06:10.305639 3590 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 6bd07faf001a401bb1f535360d89de93: Neither blocks nor log segments found. Creating new log.
I20260923 15:06:10.305930 3590 log.cc:824] T 00000000000000000000000000000000 P 6bd07faf001a401bb1f535360d89de93: Log is configured to *not* fsync() on all Append() calls
I20260923 15:06:10.306674 3590 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 6bd07faf001a401bb1f535360d89de93: No bootstrap required, opened a new log
I20260923 15:06:10.307432 13614 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 15:06:10.307744 3440 raft_consensus.cc:348] T 00000000000000000000000000000000 P e9eb8aefa7294b548c2db794e5b64efd [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9eb8aefa7294b548c2db794e5b64efd" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 40701 } } peers { permanent_uuid: "ea69d78e8d0a4f2ab829c0476e75435e" member_type: VOTER last_known_addr { host: "127.13.75.189" port: 38455 } } peers { permanent_uuid: "6bd07faf001a401bb1f535360d89de93" member_type: VOTER last_known_addr { host: "127.13.75.188" port: 38223 } }
I20260923 15:06:10.307874 3440 raft_consensus.cc:729] T 00000000000000000000000000000000 P e9eb8aefa7294b548c2db794e5b64efd [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: e9eb8aefa7294b548c2db794e5b64efd, State: Initialized, Role: FOLLOWER
I20260923 15:06:10.307950 3440 consensus_queue.cc:260] T 00000000000000000000000000000000 P e9eb8aefa7294b548c2db794e5b64efd [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9eb8aefa7294b548c2db794e5b64efd" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 40701 } } peers { permanent_uuid: "ea69d78e8d0a4f2ab829c0476e75435e" member_type: VOTER last_known_addr { host: "127.13.75.189" port: 38455 } } peers { permanent_uuid: "6bd07faf001a401bb1f535360d89de93" member_type: VOTER last_known_addr { host: "127.13.75.188" port: 38223 } }
I20260923 15:06:10.308274 3440 sys_catalog.cc:564] T 00000000000000000000000000000000 P e9eb8aefa7294b548c2db794e5b64efd [sys.catalog]: configured and running, proceeding with master startup.
I20260923 15:06:10.308230 3599 sys_catalog.cc:455] T 00000000000000000000000000000000 P e9eb8aefa7294b548c2db794e5b64efd [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9eb8aefa7294b548c2db794e5b64efd" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 40701 } } peers { permanent_uuid: "ea69d78e8d0a4f2ab829c0476e75435e" member_type: VOTER last_known_addr { host: "127.13.75.189" port: 38455 } } peers { permanent_uuid: "6bd07faf001a401bb1f535360d89de93" member_type: VOTER last_known_addr { host: "127.13.75.188" port: 38223 } } }
I20260923 15:06:10.308347 3599 sys_catalog.cc:458] T 00000000000000000000000000000000 P e9eb8aefa7294b548c2db794e5b64efd [sys.catalog]: This master's current role is: FOLLOWER
I20260923 15:06:10.308524 3590 raft_consensus.cc:348] T 00000000000000000000000000000000 P 6bd07faf001a401bb1f535360d89de93 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9eb8aefa7294b548c2db794e5b64efd" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 40701 } } peers { permanent_uuid: "ea69d78e8d0a4f2ab829c0476e75435e" member_type: VOTER last_known_addr { host: "127.13.75.189" port: 38455 } } peers { permanent_uuid: "6bd07faf001a401bb1f535360d89de93" member_type: VOTER last_known_addr { host: "127.13.75.188" port: 38223 } }
I20260923 15:06:10.308667 3590 raft_consensus.cc:729] T 00000000000000000000000000000000 P 6bd07faf001a401bb1f535360d89de93 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 6bd07faf001a401bb1f535360d89de93, State: Initialized, Role: FOLLOWER
I20260923 15:06:10.308727 3590 consensus_queue.cc:260] T 00000000000000000000000000000000 P 6bd07faf001a401bb1f535360d89de93 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9eb8aefa7294b548c2db794e5b64efd" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 40701 } } peers { permanent_uuid: "ea69d78e8d0a4f2ab829c0476e75435e" member_type: VOTER last_known_addr { host: "127.13.75.189" port: 38455 } } peers { permanent_uuid: "6bd07faf001a401bb1f535360d89de93" member_type: VOTER last_known_addr { host: "127.13.75.188" port: 38223 } }
I20260923 15:06:10.308683 3414 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "e9eb8aefa7294b548c2db794e5b64efd"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:39778
I20260923 15:06:10.308817 3414 raft_consensus.cc:482] T 00000000000000000000000000000000 P e9eb8aefa7294b548c2db794e5b64efd [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260923 15:06:10.308873 3414 raft_consensus.cc:3037] T 00000000000000000000000000000000 P e9eb8aefa7294b548c2db794e5b64efd [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:06:10.309026 3590 sys_catalog.cc:564] T 00000000000000000000000000000000 P 6bd07faf001a401bb1f535360d89de93 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 15:06:10.309870 3414 raft_consensus.cc:504] T 00000000000000000000000000000000 P e9eb8aefa7294b548c2db794e5b64efd [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9eb8aefa7294b548c2db794e5b64efd" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 40701 } } peers { permanent_uuid: "ea69d78e8d0a4f2ab829c0476e75435e" member_type: VOTER last_known_addr { host: "127.13.75.189" port: 38455 } } peers { permanent_uuid: "6bd07faf001a401bb1f535360d89de93" member_type: VOTER last_known_addr { host: "127.13.75.188" port: 38223 } }
I20260923 15:06:10.309933 3602 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6bd07faf001a401bb1f535360d89de93 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9eb8aefa7294b548c2db794e5b64efd" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 40701 } } peers { permanent_uuid: "ea69d78e8d0a4f2ab829c0476e75435e" member_type: VOTER last_known_addr { host: "127.13.75.189" port: 38455 } } peers { permanent_uuid: "6bd07faf001a401bb1f535360d89de93" member_type: VOTER last_known_addr { host: "127.13.75.188" port: 38223 } } }
I20260923 15:06:10.310007 3602 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6bd07faf001a401bb1f535360d89de93 [sys.catalog]: This master's current role is: FOLLOWER
I20260923 15:06:10.310817 3564 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e9eb8aefa7294b548c2db794e5b64efd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "6bd07faf001a401bb1f535360d89de93"
I20260923 15:06:10.310978 3564 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 6bd07faf001a401bb1f535360d89de93 [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:06:10.310148 3414 leader_election.cc:288] T 00000000000000000000000000000000 P e9eb8aefa7294b548c2db794e5b64efd [CANDIDATE]: Term 1 election: Requested vote from peers ea69d78e8d0a4f2ab829c0476e75435e (127.13.75.189:38455), 6bd07faf001a401bb1f535360d89de93 (127.13.75.188:38223)
I20260923 15:06:10.311885 3564 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 6bd07faf001a401bb1f535360d89de93 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9eb8aefa7294b548c2db794e5b64efd in term 1.
W20260923 15:06:10.311875 3620 catalog_manager.cc:1558] T 00000000000000000000000000000000 P e9eb8aefa7294b548c2db794e5b64efd: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260923 15:06:10.311931 3620 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260923 15:06:10.312058 3391 leader_election.cc:302] T 00000000000000000000000000000000 P e9eb8aefa7294b548c2db794e5b64efd [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 6bd07faf001a401bb1f535360d89de93, e9eb8aefa7294b548c2db794e5b64efd; no voters:
I20260923 15:06:10.312191 3599 raft_consensus.cc:2781] T 00000000000000000000000000000000 P e9eb8aefa7294b548c2db794e5b64efd [term 1 FOLLOWER]: Leader election won for term 1
I20260923 15:06:10.312351 3599 raft_consensus.cc:686] T 00000000000000000000000000000000 P e9eb8aefa7294b548c2db794e5b64efd [term 1 LEADER]: Becoming Leader. State: Replica: e9eb8aefa7294b548c2db794e5b64efd, State: Running, Role: LEADER
I20260923 15:06:10.312438 3599 consensus_queue.cc:237] T 00000000000000000000000000000000 P e9eb8aefa7294b548c2db794e5b64efd [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9eb8aefa7294b548c2db794e5b64efd" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 40701 } } peers { permanent_uuid: "ea69d78e8d0a4f2ab829c0476e75435e" member_type: VOTER last_known_addr { host: "127.13.75.189" port: 38455 } } peers { permanent_uuid: "6bd07faf001a401bb1f535360d89de93" member_type: VOTER last_known_addr { host: "127.13.75.188" port: 38223 } }
I20260923 15:06:10.312816 3599 sys_catalog.cc:455] T 00000000000000000000000000000000 P e9eb8aefa7294b548c2db794e5b64efd [sys.catalog]: SysCatalogTable state changed. Reason: New leader e9eb8aefa7294b548c2db794e5b64efd. Latest consensus state: current_term: 1 leader_uuid: "e9eb8aefa7294b548c2db794e5b64efd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9eb8aefa7294b548c2db794e5b64efd" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 40701 } } peers { permanent_uuid: "ea69d78e8d0a4f2ab829c0476e75435e" member_type: VOTER last_known_addr { host: "127.13.75.189" port: 38455 } } peers { permanent_uuid: "6bd07faf001a401bb1f535360d89de93" member_type: VOTER last_known_addr { host: "127.13.75.188" port: 38223 } } }
I20260923 15:06:10.312865 3599 sys_catalog.cc:458] T 00000000000000000000000000000000 P e9eb8aefa7294b548c2db794e5b64efd [sys.catalog]: This master's current role is: LEADER
I20260923 15:06:10.313951 3629 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 15:06:10.314172 3629 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 15:06:10.313410 3522 master_runner.cc:186] Error getting master registration for 127.13.75.189:38455: OK, instance_id { permanent_uuid: "ea69d78e8d0a4f2ab829c0476e75435e" instance_seqno: 1790175970108707 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260923 15:06:10.313328 3372 master_runner.cc:186] Error getting master registration for 127.13.75.189:38455: OK, instance_id { permanent_uuid: "ea69d78e8d0a4f2ab829c0476e75435e" instance_seqno: 1790175970108707 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260923 15:06:10.315794 3522 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260923 15:06:10.315798 3372 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260923 15:06:10.324674 3564 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 6bd07faf001a401bb1f535360d89de93 [term 1 FOLLOWER]: Refusing update from remote peer e9eb8aefa7294b548c2db794e5b64efd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 15:06:10.325008 3599 consensus_queue.cc:1035] T 00000000000000000000000000000000 P e9eb8aefa7294b548c2db794e5b64efd [LEADER]: Connected to new peer: Peer: permanent_uuid: "6bd07faf001a401bb1f535360d89de93" member_type: VOTER last_known_addr { host: "127.13.75.188" port: 38223 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 15:06:10.325934 3596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:10.313786 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48698 (local address 127.13.75.188:38223)
0923 15:06:10.313825 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:10.313827 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:10.316499 (+ 2672us) server_negotiation.cc:407] Connection header received
0923 15:06:10.316607 (+ 108us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:10.316607 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:10.316619 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:10.316651 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:10.317449 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:10.318195 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:10.319382 (+ 1187us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:10.319770 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:10.319845 (+ 75us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:10.321639 (+ 1794us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:06:10.321640 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:06:10.321641 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:06:10.321648 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:06:10.323461 (+ 1813us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:10.323867 (+ 406us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:10.323868 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:10.323869 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:10.323943 (+ 74us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:10.324230 (+ 287us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:10.324231 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:10.324231 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:10.324483 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:06:10.324616 (+ 133us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:10.325270 (+ 654us) server_negotiation.cc:299] Negotiation successful
0923 15:06:10.325443 (+ 173us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260923 15:06:10.326426 3595 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:10.313482 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39788 (local address 127.13.75.190:40701)
0923 15:06:10.313528 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:10.313530 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:10.314945 (+ 1415us) server_negotiation.cc:407] Connection header received
0923 15:06:10.315057 (+ 112us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:10.315058 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:10.315069 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:10.315101 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:10.315923 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:10.316688 (+ 765us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:10.318488 (+ 1800us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:10.318844 (+ 356us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:10.319046 (+ 202us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:10.323151 (+ 4105us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:06:10.323153 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:06:10.323154 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:06:10.323161 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:06:10.325143 (+ 1982us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:10.325522 (+ 379us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:10.325523 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:10.325523 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:10.325579 (+ 56us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:10.325798 (+ 219us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:10.325799 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:10.325799 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:10.325943 (+ 144us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:06:10.326031 (+ 88us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:10.326242 (+ 211us) server_negotiation.cc:299] Negotiation successful
0923 15:06:10.326325 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260923 15:06:10.326683 3602 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6bd07faf001a401bb1f535360d89de93 [sys.catalog]: SysCatalogTable state changed. Reason: New leader e9eb8aefa7294b548c2db794e5b64efd. Latest consensus state: current_term: 1 leader_uuid: "e9eb8aefa7294b548c2db794e5b64efd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9eb8aefa7294b548c2db794e5b64efd" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 40701 } } peers { permanent_uuid: "ea69d78e8d0a4f2ab829c0476e75435e" member_type: VOTER last_known_addr { host: "127.13.75.189" port: 38455 } } peers { permanent_uuid: "6bd07faf001a401bb1f535360d89de93" member_type: VOTER last_known_addr { host: "127.13.75.188" port: 38223 } } }
I20260923 15:06:10.326779 3602 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6bd07faf001a401bb1f535360d89de93 [sys.catalog]: This master's current role is: FOLLOWER
W20260923 15:06:10.328400 3623 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 6bd07faf001a401bb1f535360d89de93: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260923 15:06:10.328459 3623 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260923 15:06:10.329516 3625 sys_catalog.cc:455] T 00000000000000000000000000000000 P e9eb8aefa7294b548c2db794e5b64efd [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e9eb8aefa7294b548c2db794e5b64efd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9eb8aefa7294b548c2db794e5b64efd" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 40701 } } peers { permanent_uuid: "ea69d78e8d0a4f2ab829c0476e75435e" member_type: VOTER last_known_addr { host: "127.13.75.189" port: 38455 } } peers { permanent_uuid: "6bd07faf001a401bb1f535360d89de93" member_type: VOTER last_known_addr { host: "127.13.75.188" port: 38223 } } }
I20260923 15:06:10.329608 3625 sys_catalog.cc:458] T 00000000000000000000000000000000 P e9eb8aefa7294b548c2db794e5b64efd [sys.catalog]: This master's current role is: LEADER
I20260923 15:06:10.330063 3602 sys_catalog.cc:455] T 00000000000000000000000000000000 P 6bd07faf001a401bb1f535360d89de93 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e9eb8aefa7294b548c2db794e5b64efd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9eb8aefa7294b548c2db794e5b64efd" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 40701 } } peers { permanent_uuid: "ea69d78e8d0a4f2ab829c0476e75435e" member_type: VOTER last_known_addr { host: "127.13.75.189" port: 38455 } } peers { permanent_uuid: "6bd07faf001a401bb1f535360d89de93" member_type: VOTER last_known_addr { host: "127.13.75.188" port: 38223 } } }
I20260923 15:06:10.330163 3602 sys_catalog.cc:458] T 00000000000000000000000000000000 P 6bd07faf001a401bb1f535360d89de93 [sys.catalog]: This master's current role is: FOLLOWER
I20260923 15:06:10.330283 3629 catalog_manager.cc:1347] Generated new cluster ID: 5e36c3b1b16e48aaa3e08ef496e1d7e3
I20260923 15:06:10.330335 3629 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 15:06:10.330418 3635 mvcc.cc:205] Tried to move back new op lower bound from 7332560774425731072 to 7332560774400397312. Current Snapshot: MvccSnapshot[applied={T|T < 7332560774425731072 or (T in {7332560774425731072})}]
I20260923 15:06:10.330561 3636 mvcc.cc:205] Tried to move back new op lower bound from 7332560774425731072 to 7332560774400397312. Current Snapshot: MvccSnapshot[applied={T|T < 7332560774425731072}]
I20260923 15:06:10.336057 3519 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:10.314605 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40782 (local address 127.13.75.189:38455)
0923 15:06:10.314645 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:10.314646 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:10.319667 (+ 5021us) server_negotiation.cc:407] Connection header received
0923 15:06:10.319738 (+ 71us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:10.319740 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:10.319754 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:10.319827 (+ 73us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:10.321239 (+ 1412us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:10.321958 (+ 719us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:10.323049 (+ 1091us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:10.323376 (+ 327us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:10.323449 (+ 73us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:10.327906 (+ 4457us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:06:10.327908 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:06:10.327908 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:06:10.327916 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:06:10.329755 (+ 1839us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:10.330173 (+ 418us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:10.330174 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:10.330175 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:10.330236 (+ 61us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:10.335185 (+ 4949us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:10.335186 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:10.335186 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:10.335335 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:06:10.335429 (+ 94us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:10.335843 (+ 414us) server_negotiation.cc:299] Negotiation successful
0923 15:06:10.335932 (+ 89us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260923 15:06:10.336419 3629 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 15:06:10.337185 3629 catalog_manager.cc:1504] Loading token signing keys...
I20260923 15:06:10.341240 3629 catalog_manager.cc:5868] T 00000000000000000000000000000000 P e9eb8aefa7294b548c2db794e5b64efd: Generated new TSK 0
I20260923 15:06:10.341462 3629 catalog_manager.cc:1514] Initializing in-progress tserver states...
I20260923 15:06:10.346580 3515 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.75.188" port: 38223 } attempt: 3
Sep 23 15:06:10 dist-test-slave-cnb0 krb5kdc[3359](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175970, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.189@KRBTEST.COM for kudu/127.13.75.188@KRBTEST.COM
I20260923 15:06:10.355206 3517 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:10.346861 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.188:38223 (local address 127.0.0.1:48714)
0923 15:06:10.346964 (+ 103us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:10.346975 (+ 11us) client_negotiation.cc:174] Beginning negotiation
0923 15:06:10.347014 (+ 39us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:06:10.347312 (+ 298us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:06:10.347314 (+ 2us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:06:10.347503 (+ 189us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:06:10.347552 (+ 49us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:10.347554 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:10.348465 (+ 911us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:10.348467 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:10.349282 (+ 815us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:10.349284 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:10.349816 (+ 532us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:10.349817 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:10.349883 (+ 66us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:10.350488 (+ 605us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:06:10.350493 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:06:10.352099 (+ 1606us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:06:10.353964 (+ 1865us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:06:10.353965 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:06:10.353965 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:06:10.354180 (+ 215us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:06:10.354481 (+ 301us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:06:10.354482 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:06:10.354482 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:06:10.354540 (+ 58us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:06:10.354824 (+ 284us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:06:10.354825 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:06:10.354901 (+ 76us) client_negotiation.cc:769] Sending connection context
0923 15:06:10.354984 (+ 83us) client_negotiation.cc:240] Negotiation successful
0923 15:06:10.355094 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":85}
I20260923 15:06:10.355211 3596 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:10.347068 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48714 (local address 127.13.75.188:38223)
0923 15:06:10.347158 (+ 90us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:10.347159 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:10.347166 (+ 7us) server_negotiation.cc:407] Connection header received
0923 15:06:10.347186 (+ 20us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:10.347186 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:10.347196 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:10.347246 (+ 50us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:10.347634 (+ 388us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:10.348389 (+ 755us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:10.349377 (+ 988us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:10.349688 (+ 311us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:10.349756 (+ 68us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:10.352210 (+ 2454us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:06:10.352211 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:06:10.352211 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:06:10.352219 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:06:10.353869 (+ 1650us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:10.354333 (+ 464us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:10.354334 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:10.354335 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:10.354390 (+ 55us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:10.354611 (+ 221us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:10.354612 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:10.354613 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:10.354752 (+ 139us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:06:10.354815 (+ 63us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:10.355043 (+ 228us) server_negotiation.cc:299] Negotiation successful
0923 15:06:10.355127 (+ 84us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":68}
I20260923 15:06:10.357075 3515 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ea69d78e8d0a4f2ab829c0476e75435e: Bootstrap starting.
I20260923 15:06:10.357846 3515 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P ea69d78e8d0a4f2ab829c0476e75435e: Neither blocks nor log segments found. Creating new log.
I20260923 15:06:10.358170 3515 log.cc:824] T 00000000000000000000000000000000 P ea69d78e8d0a4f2ab829c0476e75435e: Log is configured to *not* fsync() on all Append() calls
I20260923 15:06:10.359046 3515 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P ea69d78e8d0a4f2ab829c0476e75435e: No bootstrap required, opened a new log
I20260923 15:06:10.361120 3515 raft_consensus.cc:348] T 00000000000000000000000000000000 P ea69d78e8d0a4f2ab829c0476e75435e [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9eb8aefa7294b548c2db794e5b64efd" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 40701 } } peers { permanent_uuid: "ea69d78e8d0a4f2ab829c0476e75435e" member_type: VOTER last_known_addr { host: "127.13.75.189" port: 38455 } } peers { permanent_uuid: "6bd07faf001a401bb1f535360d89de93" member_type: VOTER last_known_addr { host: "127.13.75.188" port: 38223 } }
I20260923 15:06:10.361269 3515 raft_consensus.cc:729] T 00000000000000000000000000000000 P ea69d78e8d0a4f2ab829c0476e75435e [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: ea69d78e8d0a4f2ab829c0476e75435e, State: Initialized, Role: FOLLOWER
I20260923 15:06:10.361357 3515 consensus_queue.cc:260] T 00000000000000000000000000000000 P ea69d78e8d0a4f2ab829c0476e75435e [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9eb8aefa7294b548c2db794e5b64efd" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 40701 } } peers { permanent_uuid: "ea69d78e8d0a4f2ab829c0476e75435e" member_type: VOTER last_known_addr { host: "127.13.75.189" port: 38455 } } peers { permanent_uuid: "6bd07faf001a401bb1f535360d89de93" member_type: VOTER last_known_addr { host: "127.13.75.188" port: 38223 } }
I20260923 15:06:10.361663 3645 sys_catalog.cc:455] T 00000000000000000000000000000000 P ea69d78e8d0a4f2ab829c0476e75435e [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9eb8aefa7294b548c2db794e5b64efd" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 40701 } } peers { permanent_uuid: "ea69d78e8d0a4f2ab829c0476e75435e" member_type: VOTER last_known_addr { host: "127.13.75.189" port: 38455 } } peers { permanent_uuid: "6bd07faf001a401bb1f535360d89de93" member_type: VOTER last_known_addr { host: "127.13.75.188" port: 38223 } } }
I20260923 15:06:10.361719 3489 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "e9eb8aefa7294b548c2db794e5b64efd" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "ea69d78e8d0a4f2ab829c0476e75435e"
I20260923 15:06:10.361856 3645 sys_catalog.cc:458] T 00000000000000000000000000000000 P ea69d78e8d0a4f2ab829c0476e75435e [sys.catalog]: This master's current role is: FOLLOWER
I20260923 15:06:10.361760 3488 raft_consensus.cc:3037] T 00000000000000000000000000000000 P ea69d78e8d0a4f2ab829c0476e75435e [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:06:10.362946 3488 raft_consensus.cc:1261] T 00000000000000000000000000000000 P ea69d78e8d0a4f2ab829c0476e75435e [term 1 FOLLOWER]: Refusing update from remote peer e9eb8aefa7294b548c2db794e5b64efd: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 15:06:10.363312 3599 consensus_queue.cc:1035] T 00000000000000000000000000000000 P e9eb8aefa7294b548c2db794e5b64efd [LEADER]: Connected to new peer: Peer: permanent_uuid: "ea69d78e8d0a4f2ab829c0476e75435e" member_type: VOTER last_known_addr { host: "127.13.75.189" port: 38455 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 15:06:10.363760 3489 raft_consensus.cc:2454] T 00000000000000000000000000000000 P ea69d78e8d0a4f2ab829c0476e75435e [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate e9eb8aefa7294b548c2db794e5b64efd in term 1.
I20260923 15:06:10.365025 3645 sys_catalog.cc:455] T 00000000000000000000000000000000 P ea69d78e8d0a4f2ab829c0476e75435e [sys.catalog]: SysCatalogTable state changed. Reason: New leader e9eb8aefa7294b548c2db794e5b64efd. Latest consensus state: current_term: 1 leader_uuid: "e9eb8aefa7294b548c2db794e5b64efd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9eb8aefa7294b548c2db794e5b64efd" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 40701 } } peers { permanent_uuid: "ea69d78e8d0a4f2ab829c0476e75435e" member_type: VOTER last_known_addr { host: "127.13.75.189" port: 38455 } } peers { permanent_uuid: "6bd07faf001a401bb1f535360d89de93" member_type: VOTER last_known_addr { host: "127.13.75.188" port: 38223 } } }
I20260923 15:06:10.365119 3645 sys_catalog.cc:458] T 00000000000000000000000000000000 P ea69d78e8d0a4f2ab829c0476e75435e [sys.catalog]: This master's current role is: FOLLOWER
I20260923 15:06:10.365161 3645 sys_catalog.cc:455] T 00000000000000000000000000000000 P ea69d78e8d0a4f2ab829c0476e75435e [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "e9eb8aefa7294b548c2db794e5b64efd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9eb8aefa7294b548c2db794e5b64efd" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 40701 } } peers { permanent_uuid: "ea69d78e8d0a4f2ab829c0476e75435e" member_type: VOTER last_known_addr { host: "127.13.75.189" port: 38455 } } peers { permanent_uuid: "6bd07faf001a401bb1f535360d89de93" member_type: VOTER last_known_addr { host: "127.13.75.188" port: 38223 } } }
I20260923 15:06:10.365214 3645 sys_catalog.cc:458] T 00000000000000000000000000000000 P ea69d78e8d0a4f2ab829c0476e75435e [sys.catalog]: This master's current role is: FOLLOWER
I20260923 15:06:10.361694 3515 sys_catalog.cc:564] T 00000000000000000000000000000000 P ea69d78e8d0a4f2ab829c0476e75435e [sys.catalog]: configured and running, proceeding with master startup.
W20260923 15:06:10.367141 3660 catalog_manager.cc:1558] T 00000000000000000000000000000000 P ea69d78e8d0a4f2ab829c0476e75435e: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260923 15:06:10.367244 3660 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260923 15:06:10.367709 3647 mvcc.cc:205] Tried to move back new op lower bound from 7332560774513922048 to 7332560774400397312. Current Snapshot: MvccSnapshot[applied={T|T < 7332560774425731072}]
W20260923 15:06:10.376293 3596 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 15:06:10.375152 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48724 (local address 127.13.75.188:38223)
0923 15:06:10.375191 (+ 39us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:10.375193 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:10.375722 (+ 529us) server_negotiation.cc:407] Connection header received
0923 15:06:10.375822 (+ 100us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:10.375823 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:10.375834 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:10.375866 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:10.376206 (+ 340us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48724: BlockingRecv error: recv got EOF from 127.0.0.1:48724 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260923 15:06:10.376338 3595 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 15:06:10.374911 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39804 (local address 127.13.75.190:40701)
0923 15:06:10.374955 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:10.374958 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:10.375618 (+ 660us) server_negotiation.cc:407] Connection header received
0923 15:06:10.375646 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:10.375647 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:10.375659 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:10.375711 (+ 52us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:10.376275 (+ 564us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:39804: BlockingRecv error: recv got EOF from 127.0.0.1:39804 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260923 15:06:10.376667 3519 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 15:06:10.374441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40796 (local address 127.13.75.189:38455)
0923 15:06:10.374490 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:10.374493 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:10.375424 (+ 931us) server_negotiation.cc:407] Connection header received
0923 15:06:10.375503 (+ 79us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:10.375505 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:10.375516 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:10.375574 (+ 58us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:10.376606 (+ 1032us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40796: BlockingRecv error: recv got EOF from 127.0.0.1:40796 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260923 15:06:10.381313 3519 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 15:06:10.380502 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40798 (local address 127.13.75.189:38455)
0923 15:06:10.380542 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:10.380544 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:10.380781 (+ 237us) server_negotiation.cc:407] Connection header received
0923 15:06:10.380888 (+ 107us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:10.380889 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:10.380907 (+ 18us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:10.380936 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:10.381261 (+ 325us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40798: BlockingRecv error: recv got EOF from 127.0.0.1:40798 (error 108)
Metrics: {"server-negotiator.queue_time_us":19}
W20260923 15:06:10.384511 3519 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 15:06:10.383696 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40804 (local address 127.13.75.189:38455)
0923 15:06:10.383730 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:10.383732 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:10.384018 (+ 286us) server_negotiation.cc:407] Connection header received
0923 15:06:10.384114 (+ 96us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:10.384115 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:10.384125 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:10.384154 (+ 29us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:10.384463 (+ 309us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40804: BlockingRecv error: recv got EOF from 127.0.0.1:40804 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260923 15:06:10.388593 3519 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 15:06:10.387872 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40810 (local address 127.13.75.189:38455)
0923 15:06:10.387906 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:10.387907 (+ 1us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:10.388100 (+ 193us) server_negotiation.cc:407] Connection header received
0923 15:06:10.388197 (+ 97us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:10.388198 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:10.388209 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:10.388237 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:10.388547 (+ 310us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40810: BlockingRecv error: recv got EOF from 127.0.0.1:40810 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260923 15:06:10.394558 3519 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 15:06:10.393832 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40816 (local address 127.13.75.189:38455)
0923 15:06:10.393864 (+ 32us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:10.393866 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:10.394063 (+ 197us) server_negotiation.cc:407] Connection header received
0923 15:06:10.394158 (+ 95us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:10.394159 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:10.394169 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:10.394199 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:10.394508 (+ 309us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40816: BlockingRecv error: recv got EOF from 127.0.0.1:40816 (error 108)
Metrics: {"server-negotiator.queue_time_us":16}
W20260923 15:06:10.405043 3519 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 15:06:10.404175 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40824 (local address 127.13.75.189:38455)
0923 15:06:10.404217 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:10.404219 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:10.404455 (+ 236us) server_negotiation.cc:407] Connection header received
0923 15:06:10.404561 (+ 106us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:10.404562 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:10.404573 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:10.404617 (+ 44us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:10.404983 (+ 366us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40824: BlockingRecv error: recv got EOF from 127.0.0.1:40824 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260923 15:06:10.423432 3519 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 15:06:10.422488 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40836 (local address 127.13.75.189:38455)
0923 15:06:10.422532 (+ 44us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:10.422534 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:10.422793 (+ 259us) server_negotiation.cc:407] Connection header received
0923 15:06:10.422937 (+ 144us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:10.422938 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:10.422951 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:10.422984 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:10.423369 (+ 385us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40836: BlockingRecv error: recv got EOF from 127.0.0.1:40836 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260923 15:06:10.457962 3519 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 15:06:10.456995 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40852 (local address 127.13.75.189:38455)
0923 15:06:10.457042 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:10.457044 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:10.457319 (+ 275us) server_negotiation.cc:407] Connection header received
0923 15:06:10.457433 (+ 114us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:10.457435 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:10.457448 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:10.457480 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:10.457897 (+ 417us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40852: BlockingRecv error: recv got EOF from 127.0.0.1:40852 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260923 15:06:10.524199 3519 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 15:06:10.523451 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40854 (local address 127.13.75.189:38455)
0923 15:06:10.523503 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:10.523505 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:10.523637 (+ 132us) server_negotiation.cc:407] Connection header received
0923 15:06:10.523723 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:10.523724 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:10.523736 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:10.523803 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:10.524131 (+ 328us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40854: BlockingRecv error: recv got EOF from 127.0.0.1:40854 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
W20260923 15:06:10.654382 3519 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 15:06:10.653577 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40866 (local address 127.13.75.189:38455)
0923 15:06:10.653687 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:10.653690 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:10.653808 (+ 118us) server_negotiation.cc:407] Connection header received
0923 15:06:10.653889 (+ 81us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:10.653890 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:10.653903 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:10.653957 (+ 54us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:10.654317 (+ 360us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40866: BlockingRecv error: recv got EOF from 127.0.0.1:40866 (error 108)
Metrics: {"server-negotiator.queue_time_us":74}
W20260923 15:06:10.912281 3519 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 15:06:10.911694 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40870 (local address 127.13.75.189:38455)
0923 15:06:10.911759 (+ 65us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:10.911761 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:10.911773 (+ 12us) server_negotiation.cc:407] Connection header received
0923 15:06:10.911832 (+ 59us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:10.911833 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:10.911843 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:10.911914 (+ 71us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:10.912208 (+ 294us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:40870: BlockingRecv error: recv got EOF from 127.0.0.1:40870 (error 108)
Metrics: {"server-negotiator.queue_time_us":35}
I20260923 15:06:10.940693 3599 sys_catalog.cc:455] T 00000000000000000000000000000000 P e9eb8aefa7294b548c2db794e5b64efd [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "e9eb8aefa7294b548c2db794e5b64efd" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "e9eb8aefa7294b548c2db794e5b64efd" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 40701 } } peers { permanent_uuid: "ea69d78e8d0a4f2ab829c0476e75435e" member_type: VOTER last_known_addr { host: "127.13.75.189" port: 38455 } } peers { permanent_uuid: "6bd07faf001a401bb1f535360d89de93" member_type: VOTER last_known_addr { host: "127.13.75.188" port: 38223 } } }
I20260923 15:06:10.940850 3599 sys_catalog.cc:458] T 00000000000000000000000000000000 P e9eb8aefa7294b548c2db794e5b64efd [sys.catalog]: This master's current role is: LEADER
I20260923 15:06:11.328941 3623 catalog_manager.cc:1259] Loaded cluster ID: 5e36c3b1b16e48aaa3e08ef496e1d7e3
I20260923 15:06:11.329064 3623 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 6bd07faf001a401bb1f535360d89de93: loading cluster ID for follower catalog manager: success
I20260923 15:06:11.329864 3623 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 6bd07faf001a401bb1f535360d89de93: acquiring CA information for follower catalog manager: success
I20260923 15:06:11.330118 3623 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 6bd07faf001a401bb1f535360d89de93: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260923 15:06:11.367763 3660 catalog_manager.cc:1259] Loaded cluster ID: 5e36c3b1b16e48aaa3e08ef496e1d7e3
I20260923 15:06:11.367841 3660 catalog_manager.cc:1552] T 00000000000000000000000000000000 P ea69d78e8d0a4f2ab829c0476e75435e: loading cluster ID for follower catalog manager: success
I20260923 15:06:11.368774 3660 catalog_manager.cc:1574] T 00000000000000000000000000000000 P ea69d78e8d0a4f2ab829c0476e75435e: acquiring CA information for follower catalog manager: success
I20260923 15:06:11.369056 3660 catalog_manager.cc:1602] T 00000000000000000000000000000000 P ea69d78e8d0a4f2ab829c0476e75435e: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260923 15:06:11.428134 3730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:11.425498 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40884 (local address 127.13.75.189:38455)
0923 15:06:11.425649 (+ 151us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:11.425651 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:11.425685 (+ 34us) server_negotiation.cc:407] Connection header received
0923 15:06:11.425712 (+ 27us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:11.425712 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:11.425766 (+ 54us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:11.425832 (+ 66us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 15:06:11.426051 (+ 219us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:11.426564 (+ 513us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:11.427362 (+ 798us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:11.427577 (+ 215us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:11.427646 (+ 69us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:11.427787 (+ 141us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 15:06:11.427901 (+ 114us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 15:06:11.427959 (+ 58us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:11.428003 (+ 44us) server_negotiation.cc:299] Negotiation successful
0923 15:06:11.428063 (+ 60us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":128,"thread_start_us":87,"threads_started":1}
I20260923 15:06:11.432308 3736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:11.429959 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48730 (local address 127.13.75.188:38223)
0923 15:06:11.430070 (+ 111us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:11.430072 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:11.430080 (+ 8us) server_negotiation.cc:407] Connection header received
0923 15:06:11.430117 (+ 37us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:11.430117 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:11.430183 (+ 66us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:11.430255 (+ 72us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 15:06:11.430341 (+ 86us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:11.430827 (+ 486us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:11.431602 (+ 775us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:11.431805 (+ 203us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:11.431879 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:11.432013 (+ 134us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 15:06:11.432123 (+ 110us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 15:06:11.432182 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:11.432224 (+ 42us) server_negotiation.cc:299] Negotiation successful
0923 15:06:11.432242 (+ 18us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":92,"thread_start_us":65,"threads_started":1}
I20260923 15:06:11.437497 3730 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:11.434107 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:40886 (local address 127.13.75.189:38455)
0923 15:06:11.434156 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:11.434158 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:11.434420 (+ 262us) server_negotiation.cc:407] Connection header received
0923 15:06:11.434481 (+ 61us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:11.434482 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:11.434495 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:11.434544 (+ 49us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 15:06:11.434727 (+ 183us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:11.435473 (+ 746us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:11.436685 (+ 1212us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:11.436986 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:11.437051 (+ 65us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:11.437231 (+ 180us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 15:06:11.437263 (+ 32us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 15:06:11.437311 (+ 48us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:11.437369 (+ 58us) server_negotiation.cc:299] Negotiation successful
0923 15:06:11.437392 (+ 23us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":27}
I20260923 15:06:11.437963 3736 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:11.434406 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48746 (local address 127.13.75.188:38223)
0923 15:06:11.434444 (+ 38us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:11.434446 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:11.434573 (+ 127us) server_negotiation.cc:407] Connection header received
0923 15:06:11.434641 (+ 68us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:11.434642 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:11.434653 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:11.434707 (+ 54us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 15:06:11.435809 (+ 1102us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:11.436248 (+ 439us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:11.437192 (+ 944us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:11.437517 (+ 325us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:11.437584 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:11.437723 (+ 139us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 15:06:11.437754 (+ 31us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 15:06:11.437801 (+ 47us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:11.437865 (+ 64us) server_negotiation.cc:299] Negotiation successful
0923 15:06:11.437885 (+ 20us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260923 15:06:11.438233 3743 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:11.434144 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:39820 (local address 127.13.75.190:40701)
0923 15:06:11.434254 (+ 110us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:11.434256 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:11.434265 (+ 9us) server_negotiation.cc:407] Connection header received
0923 15:06:11.434359 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:11.434360 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:11.434412 (+ 52us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:11.434495 (+ 83us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 15:06:11.434643 (+ 148us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:11.435463 (+ 820us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:11.437092 (+ 1629us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:11.437399 (+ 307us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:11.437736 (+ 337us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:11.437815 (+ 79us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 15:06:11.437930 (+ 115us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 15:06:11.437994 (+ 64us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:11.438125 (+ 131us) server_negotiation.cc:299] Negotiation successful
0923 15:06:11.438154 (+ 29us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":86,"thread_start_us":51,"threads_started":1}
I20260923 15:06:11.440253 13614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKC4soy/build/release/bin/kudu with pid 3372
I20260923 15:06:11.445948 13614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKC4soy/build/release/bin/kudu with pid 3447
I20260923 15:06:11.451696 13614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKC4soy/build/release/bin/kudu with pid 3522
2026-09-23T15:06:11Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/0 (2819 ms)
[ RUN ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1
Loading random data
Initializing database '/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790175691491353-13614-0/krb5kdc/principal' for realm 'KRBTEST.COM',
master key name 'K/M@KRBTEST.COM'
Sep 23 15:06:11 dist-test-slave-cnb0 krb5kdc[3747](info): setting up network...
krb5kdc: setsockopt(10,IPV6_V6ONLY,1) worked
Sep 23 15:06:11 dist-test-slave-cnb0 krb5kdc[3747](info): set up 2 sockets
Sep 23 15:06:11 dist-test-slave-cnb0 krb5kdc[3747](info): commencing operation
krb5kdc: starting...
W20260923 15:06:12.508334 13614 mini_kdc.cc:121] Time spent starting KDC: real 1.035s user 0.002s sys 0.004s
WARNING: no policy specified for test-admin@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-admin@KRBTEST.COM" created.
WARNING: no policy specified for test-user@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "test-user@KRBTEST.COM" created.
WARNING: no policy specified for joe-interloper@KRBTEST.COM; defaulting to no policy
Authenticating as principal slave/admin@KRBTEST.COM with password.
Principal "joe-interloper@KRBTEST.COM" created.
Authenticating as principal slave/admin@KRBTEST.COM with password.
Entry for principal test-user with kvno 1, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790175691491353-13614-0/krb5kdc/test-user.keytab.
Entry for principal test-user with kvno 1, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790175691491353-13614-0/krb5kdc/test-user.keytab.
Sep 23 15:06:12 dist-test-slave-cnb0 krb5kdc[3747](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175972, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
Password for test-admin@KRBTEST.COM:
2026-09-23T15:06:12Z chronyd version 3.5 starting (+CMDMON +NTP +REFCLOCK +RTC -PRIVDROP -SCFILTER -SIGND +ASYNCDNS -SECHASH -IPV6 +DEBUG)
2026-09-23T15:06:12Z Disabled control of system clock
WARNING: no policy specified for kudu/127.13.75.190@KRBTEST.COM; defaulting to no policy
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.75.190@KRBTEST.COM" created.
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.75.190 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.190.keytab.
Entry for principal kudu/127.13.75.190 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.190.keytab.
I20260923 15:06:12.635982 13614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKC4soy/build/release/bin/kudu
/tmp/dist-test-taskKC4soy/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790175691491353-13614-0/minicluster-data/master-0/wal
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790175691491353-13614-0/minicluster-data/master-0/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790175691491353-13614-0/minicluster-data/master-0/logs
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790175691491353-13614-0/minicluster-data/master-0/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.75.190:42059
--webserver_interface=127.13.75.190
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.190.keytab
--principal=kudu/127.13.75.190
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.75.148:42955
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.75.190:42059,127.13.75.189:39767,127.13.75.188:34245
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790175691491353-13614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790175691491353-13614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790175691491353-13614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
W20260923 15:06:12.718031 3760 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260923 15:06:12.718309 3760 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:06:12.718340 3760 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:06:12.718385 3760 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:06:12.719658 3760 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 15:06:12.719732 3760 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:06:12.719758 3760 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:06:12.719780 3760 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 15:06:12.719807 3760 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 15:06:12.721414 3760 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.75.148:42955
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790175691491353-13614-0/minicluster-data/master-0/data
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790175691491353-13614-0/minicluster-data/master-0/wal
--ipki_ca_key_size=768
--master_addresses=127.13.75.190:42059,127.13.75.189:39767,127.13.75.188:34245
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.190.keytab
--principal=kudu/127.13.75.190
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.75.190:42059
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790175691491353-13614-0/minicluster-data/master-0/data/info.pb
--webserver_interface=127.13.75.190
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3760
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790175691491353-13614-0/minicluster-data/master-0/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:06:12.721726 3760 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:06:12.722003 3760 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:06:12.728192 3766 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:06:12.728192 3768 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:06:12.728230 3765 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:06:12.728657 3760 server_base.cc:1034] running on GCE node
I20260923 15:06:12.728873 3760 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:06:12.729133 3760 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:06:12.730283 3760 hybrid_clock.cc:648] HybridClock initialized: now 1790175972730263 us; error 27 us; skew 500 ppm
Sep 23 15:06:12 dist-test-slave-cnb0 krb5kdc[3747](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175972, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.190@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:06:12.739274 3760 init.cc:377] Logged in from keytab as kudu/127.13.75.190@KRBTEST.COM (short username kudu)
I20260923 15:06:12.739810 3760 webserver.cc:460] Webserver started at http://127.13.75.190:45119/ using document root <none> and password file <none>
I20260923 15:06:12.740044 3760 fs_manager.cc:359] Metadata directory not provided
I20260923 15:06:12.740090 3760 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:06:12.740231 3760 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:06:12.741331 3760 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790175691491353-13614-0/minicluster-data/master-0/data/instance:
uuid: "925e4dec28a444a0b61b5bcb77464ce1"
format_stamp: "Formatted at 2026-09-23 15:06:12 on dist-test-slave-cnb0"
server_key: "cd530f906f9ae1698d88ad51561422b9"
server_key_iv: "f4531be5e84a293d64212c8d324ab0b6"
server_key_version: "encryptionkey@0"
I20260923 15:06:12.741700 3760 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790175691491353-13614-0/minicluster-data/master-0/wal/instance:
uuid: "925e4dec28a444a0b61b5bcb77464ce1"
format_stamp: "Formatted at 2026-09-23 15:06:12 on dist-test-slave-cnb0"
server_key: "cd530f906f9ae1698d88ad51561422b9"
server_key_iv: "f4531be5e84a293d64212c8d324ab0b6"
server_key_version: "encryptionkey@0"
I20260923 15:06:12.743374 3760 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.003s sys 0.000s
I20260923 15:06:12.744441 3775 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:06:12.744642 3760 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:06:12.744755 3760 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790175691491353-13614-0/minicluster-data/master-0/data,/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790175691491353-13614-0/minicluster-data/master-0/wal
uuid: "925e4dec28a444a0b61b5bcb77464ce1"
format_stamp: "Formatted at 2026-09-23 15:06:12 on dist-test-slave-cnb0"
server_key: "cd530f906f9ae1698d88ad51561422b9"
server_key_iv: "f4531be5e84a293d64212c8d324ab0b6"
server_key_version: "encryptionkey@0"
I20260923 15:06:12.744853 3760 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790175691491353-13614-0/minicluster-data/master-0/wal
metadata directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790175691491353-13614-0/minicluster-data/master-0/wal
1 data directories: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790175691491353-13614-0/minicluster-data/master-0/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:06:12.762226 3760 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:06:12.764385 3760 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:06:12.764551 3760 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:06:12.768991 3760 rpc_server.cc:307] RPC server started. Bound to: 127.13.75.190:42059
I20260923 15:06:12.769043 3827 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.75.190:42059 every 8 connection(s)
I20260923 15:06:12.769433 3760 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790175691491353-13614-0/minicluster-data/master-0/data/info.pb
I20260923 15:06:12.770447 3828 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:06:12.771450 13614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKC4soy/build/release/bin/kudu as pid 3760
I20260923 15:06:12.771553 13614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790175691491353-13614-0/minicluster-data/master-0/wal/instance
I20260923 15:06:12.771762 13614 external_mini_cluster.cc:1428] Setting key e77925ba45b0cb43a7a2877b7c3e0893
I20260923 15:06:12.772658 3828 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.75.190" port: 42059 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.13.75.189@KRBTEST.COM; defaulting to no policy
Sep 23 15:06:12 dist-test-slave-cnb0 krb5kdc[3747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175972, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.190@KRBTEST.COM for kudu/127.13.75.190@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.75.189@KRBTEST.COM" created.
I20260923 15:06:12.783334 3832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:12.773866 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52010 (local address 127.13.75.190:42059)
0923 15:06:12.774011 (+ 145us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:12.774016 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:12.774060 (+ 44us) server_negotiation.cc:407] Connection header received
0923 15:06:12.774620 (+ 560us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:12.774623 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:12.774645 (+ 22us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:12.774759 (+ 114us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:12.775431 (+ 672us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:12.776321 (+ 890us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:12.777240 (+ 919us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:12.777570 (+ 330us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:12.777699 (+ 129us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:12.779900 (+ 2201us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:06:12.779908 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:06:12.779908 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:06:12.779916 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:06:12.781614 (+ 1698us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:12.782018 (+ 404us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:12.782021 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:12.782021 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:12.782163 (+ 142us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:12.782392 (+ 229us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:12.782392 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:12.782393 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:12.782671 (+ 278us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:06:12.782770 (+ 99us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:12.783021 (+ 251us) server_negotiation.cc:299] Negotiation successful
0923 15:06:12.783134 (+ 113us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":116,"thread_start_us":64,"threads_started":1}
I20260923 15:06:12.783432 3831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:12.773617 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:42059 (local address 127.0.0.1:52010)
0923 15:06:12.773853 (+ 236us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:12.773895 (+ 42us) client_negotiation.cc:174] Beginning negotiation
0923 15:06:12.774447 (+ 552us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:06:12.774764 (+ 317us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:06:12.774767 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:06:12.775154 (+ 387us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:06:12.775327 (+ 173us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:12.775332 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:12.776437 (+ 1105us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:12.776438 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:12.777141 (+ 703us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:12.777143 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:12.777684 (+ 541us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:12.777685 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:12.777750 (+ 65us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:12.778317 (+ 567us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:06:12.778326 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:06:12.779806 (+ 1480us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:06:12.781714 (+ 1908us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:06:12.781721 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:06:12.781723 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:06:12.781922 (+ 199us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:06:12.782237 (+ 315us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:06:12.782238 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:06:12.782238 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:06:12.782326 (+ 88us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:06:12.782747 (+ 421us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:06:12.782753 (+ 6us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:06:12.782821 (+ 68us) client_negotiation.cc:769] Sending connection context
0923 15:06:12.782951 (+ 130us) client_negotiation.cc:240] Negotiation successful
0923 15:06:12.783215 (+ 264us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":187,"mutex_wait_us":58,"thread_start_us":120,"threads_started":1}
I20260923 15:06:12.784250 3828 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.75.189" port: 39767 } has no permanent_uuid. Determining permanent_uuid...
I20260923 15:06:12.784682 3831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:12.784544 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.189:39767 (local address 127.0.0.1:48258)
0923 15:06:12.784615 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:12.784660 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.189:39767: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":30}
W20260923 15:06:12.784821 3779 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.13.75.189:39767: connect: Connection refused (error 111)
I20260923 15:06:12.785316 3831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:12.785216 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.189:39767 (local address 127.0.0.1:48270)
0923 15:06:12.785263 (+ 47us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:12.785288 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.189:39767: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260923 15:06:12.785411 3828 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.75.189:39767: Network error: Client connection negotiation failed: client connection to 127.13.75.189:39767: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.75.189 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.189.keytab.
Entry for principal kudu/127.13.75.189 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.189.keytab.
I20260923 15:06:12.799278 13614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKC4soy/build/release/bin/kudu
/tmp/dist-test-taskKC4soy/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790175691491353-13614-0/minicluster-data/master-1/wal
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790175691491353-13614-0/minicluster-data/master-1/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790175691491353-13614-0/minicluster-data/master-1/logs
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790175691491353-13614-0/minicluster-data/master-1/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.75.189:39767
--webserver_interface=127.13.75.189
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.189.keytab
--principal=kudu/127.13.75.189
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.75.148:42955
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.75.190:42059,127.13.75.189:39767,127.13.75.188:34245
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790175691491353-13614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790175691491353-13614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790175691491353-13614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260923 15:06:12.834677 3828 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.75.189" port: 39767 } attempt: 1
I20260923 15:06:12.835290 3831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:12.835089 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.189:39767 (local address 127.0.0.1:48284)
0923 15:06:12.835182 (+ 93us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:12.835241 (+ 59us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.189:39767: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":59}
I20260923 15:06:12.835807 3831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:12.835677 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.189:39767 (local address 127.0.0.1:48288)
0923 15:06:12.835732 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:12.835771 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.189:39767: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260923 15:06:12.835959 3828 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.75.189:39767: Network error: Client connection negotiation failed: client connection to 127.13.75.189:39767: connect: Connection refused (error 111)
W20260923 15:06:12.882644 3835 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260923 15:06:12.882869 3835 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:06:12.882915 3835 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:06:12.882937 3835 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:06:12.884152 3835 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 15:06:12.884198 3835 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:06:12.884213 3835 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:06:12.884227 3835 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 15:06:12.884275 3835 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 15:06:12.885828 3835 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.75.148:42955
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790175691491353-13614-0/minicluster-data/master-1/data
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790175691491353-13614-0/minicluster-data/master-1/wal
--ipki_ca_key_size=768
--master_addresses=127.13.75.190:42059,127.13.75.189:39767,127.13.75.188:34245
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.189.keytab
--principal=kudu/127.13.75.189
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.75.189:39767
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790175691491353-13614-0/minicluster-data/master-1/data/info.pb
--webserver_interface=127.13.75.189
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3835
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790175691491353-13614-0/minicluster-data/master-1/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:06:12.886098 3835 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:06:12.886338 3835 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:06:12.891476 3840 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:06:12.891535 3835 server_base.cc:1034] running on GCE node
W20260923 15:06:12.891470 3843 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:06:12.891470 3841 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:06:12.891949 3835 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:06:12.892251 3835 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:06:12.893405 3835 hybrid_clock.cc:648] HybridClock initialized: now 1790175972893389 us; error 37 us; skew 500 ppm
Sep 23 15:06:12 dist-test-slave-cnb0 krb5kdc[3747](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175972, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.189@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:06:12.895967 3835 init.cc:377] Logged in from keytab as kudu/127.13.75.189@KRBTEST.COM (short username kudu)
I20260923 15:06:12.896382 3835 webserver.cc:460] Webserver started at http://127.13.75.189:42745/ using document root <none> and password file <none>
I20260923 15:06:12.896646 3835 fs_manager.cc:359] Metadata directory not provided
I20260923 15:06:12.896694 3835 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:06:12.896854 3835 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:06:12.897929 3835 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790175691491353-13614-0/minicluster-data/master-1/data/instance:
uuid: "33e89a99a9f146cb93c7445c741be846"
format_stamp: "Formatted at 2026-09-23 15:06:12 on dist-test-slave-cnb0"
server_key: "0d16c0b1182676f4be3e8176d36648c1"
server_key_iv: "ede5eedd097669d6bad6b80c72a71684"
server_key_version: "encryptionkey@0"
I20260923 15:06:12.898293 3835 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790175691491353-13614-0/minicluster-data/master-1/wal/instance:
uuid: "33e89a99a9f146cb93c7445c741be846"
format_stamp: "Formatted at 2026-09-23 15:06:12 on dist-test-slave-cnb0"
server_key: "0d16c0b1182676f4be3e8176d36648c1"
server_key_iv: "ede5eedd097669d6bad6b80c72a71684"
server_key_version: "encryptionkey@0"
I20260923 15:06:12.899784 3835 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.000s sys 0.003s
I20260923 15:06:12.900605 3850 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:06:12.900831 3835 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:06:12.900923 3835 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790175691491353-13614-0/minicluster-data/master-1/data,/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790175691491353-13614-0/minicluster-data/master-1/wal
uuid: "33e89a99a9f146cb93c7445c741be846"
format_stamp: "Formatted at 2026-09-23 15:06:12 on dist-test-slave-cnb0"
server_key: "0d16c0b1182676f4be3e8176d36648c1"
server_key_iv: "ede5eedd097669d6bad6b80c72a71684"
server_key_version: "encryptionkey@0"
I20260923 15:06:12.901005 3835 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790175691491353-13614-0/minicluster-data/master-1/wal
metadata directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790175691491353-13614-0/minicluster-data/master-1/wal
1 data directories: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790175691491353-13614-0/minicluster-data/master-1/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:06:12.904155 3828 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.75.189" port: 39767 } attempt: 2
I20260923 15:06:12.904620 3831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:12.904487 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.189:39767 (local address 127.0.0.1:48290)
0923 15:06:12.904547 (+ 60us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:12.904587 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.189:39767: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":36}
I20260923 15:06:12.904983 3831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:12.904878 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.189:39767 (local address 127.0.0.1:48302)
0923 15:06:12.904924 (+ 46us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:12.904952 (+ 28us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.189:39767: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260923 15:06:12.905066 3828 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.75.189:39767: Network error: Client connection negotiation failed: client connection to 127.13.75.189:39767: connect: Connection refused (error 111)
I20260923 15:06:12.917723 3835 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:06:12.920055 3835 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:06:12.920234 3835 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:06:12.924584 3835 rpc_server.cc:307] RPC server started. Bound to: 127.13.75.189:39767
I20260923 15:06:12.924641 3902 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.75.189:39767 every 8 connection(s)
I20260923 15:06:12.925030 3835 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790175691491353-13614-0/minicluster-data/master-1/data/info.pb
I20260923 15:06:12.926070 3903 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:06:12.927764 3903 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.75.190" port: 42059 } has no permanent_uuid. Determining permanent_uuid...
Sep 23 15:06:12 dist-test-slave-cnb0 krb5kdc[3747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175972, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.189@KRBTEST.COM for kudu/127.13.75.190@KRBTEST.COM
I20260923 15:06:12.934564 13614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKC4soy/build/release/bin/kudu as pid 3835
I20260923 15:06:12.934679 13614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790175691491353-13614-0/minicluster-data/master-1/wal/instance
I20260923 15:06:12.934933 13614 external_mini_cluster.cc:1428] Setting key 273cea9b320c5cde9414ab5cf94c62eb
I20260923 15:06:12.937453 3832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:12.928508 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52020 (local address 127.13.75.190:42059)
0923 15:06:12.928571 (+ 63us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:12.928573 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:12.928809 (+ 236us) server_negotiation.cc:407] Connection header received
0923 15:06:12.929344 (+ 535us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:12.929346 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:12.929359 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:12.929419 (+ 60us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:12.930046 (+ 627us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:12.930554 (+ 508us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:12.931635 (+ 1081us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:12.931848 (+ 213us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:12.931924 (+ 76us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:12.934083 (+ 2159us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:06:12.934085 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:06:12.934086 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:06:12.934095 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:06:12.935793 (+ 1698us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:12.936221 (+ 428us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:12.936222 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:12.936223 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:12.936289 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:12.936681 (+ 392us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:12.936681 (+ 0us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:12.936682 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:12.936806 (+ 124us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:06:12.936865 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:12.937308 (+ 443us) server_negotiation.cc:299] Negotiation successful
0923 15:06:12.937386 (+ 78us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
I20260923 15:06:12.937799 3905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:12.928497 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:42059 (local address 127.0.0.1:52020)
0923 15:06:12.928674 (+ 177us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:12.928718 (+ 44us) client_negotiation.cc:174] Beginning negotiation
0923 15:06:12.929168 (+ 450us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:06:12.929492 (+ 324us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:06:12.929495 (+ 3us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:06:12.929757 (+ 262us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:06:12.929954 (+ 197us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:12.929958 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:12.930643 (+ 685us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:12.930644 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:12.931543 (+ 899us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:12.931545 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:12.931950 (+ 405us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:12.931951 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:12.932049 (+ 98us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:12.932624 (+ 575us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:06:12.932633 (+ 9us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:06:12.933980 (+ 1347us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:06:12.935896 (+ 1916us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:06:12.935903 (+ 7us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:06:12.935905 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:06:12.936135 (+ 230us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:06:12.936396 (+ 261us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:06:12.936398 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:06:12.936398 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:06:12.936585 (+ 187us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:06:12.936898 (+ 313us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:06:12.936908 (+ 10us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:06:12.937121 (+ 213us) client_negotiation.cc:769] Sending connection context
0923 15:06:12.937271 (+ 150us) client_negotiation.cc:240] Negotiation successful
0923 15:06:12.937484 (+ 213us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":138,"thread_start_us":95,"threads_started":1}
I20260923 15:06:12.938417 3903 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.75.189" port: 39767 } has no permanent_uuid. Determining permanent_uuid...
WARNING: no policy specified for kudu/127.13.75.188@KRBTEST.COM; defaulting to no policy
Sep 23 15:06:12 dist-test-slave-cnb0 krb5kdc[3747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175972, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.189@KRBTEST.COM for kudu/127.13.75.189@KRBTEST.COM
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Principal "kudu/127.13.75.188@KRBTEST.COM" created.
I20260923 15:06:12.947499 3907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:12.938828 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48316 (local address 127.13.75.189:39767)
0923 15:06:12.939023 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:12.939028 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:12.939056 (+ 28us) server_negotiation.cc:407] Connection header received
0923 15:06:12.939080 (+ 24us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:12.939082 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:12.939096 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:12.939182 (+ 86us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:12.939517 (+ 335us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:12.940721 (+ 1204us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:12.941435 (+ 714us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:12.941944 (+ 509us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:12.942022 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:12.944063 (+ 2041us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:06:12.944073 (+ 10us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:06:12.944074 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:06:12.944082 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:06:12.945986 (+ 1904us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:12.946394 (+ 408us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:12.946397 (+ 3us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:12.946397 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:12.946473 (+ 76us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:12.946679 (+ 206us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:12.946680 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:12.946680 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:12.946934 (+ 254us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:06:12.947029 (+ 95us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:12.947213 (+ 184us) server_negotiation.cc:299] Negotiation successful
0923 15:06:12.947323 (+ 110us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":157,"thread_start_us":121,"threads_started":1}
I20260923 15:06:12.947505 3905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:12.938744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.189:39767 (local address 127.0.0.1:48316)
0923 15:06:12.938800 (+ 56us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:12.938814 (+ 14us) client_negotiation.cc:174] Beginning negotiation
0923 15:06:12.938862 (+ 48us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:06:12.939185 (+ 323us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:06:12.939186 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:06:12.939345 (+ 159us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:06:12.939425 (+ 80us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:12.939427 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:12.940813 (+ 1386us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:12.940814 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:12.941354 (+ 540us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:12.941356 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:12.942035 (+ 679us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:12.942036 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:12.942098 (+ 62us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:12.942593 (+ 495us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:06:12.942598 (+ 5us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:06:12.943978 (+ 1380us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:06:12.946076 (+ 2098us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:06:12.946077 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:06:12.946078 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:06:12.946315 (+ 237us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:06:12.946543 (+ 228us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:06:12.946544 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:06:12.946544 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:06:12.946607 (+ 63us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:06:12.947043 (+ 436us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:06:12.947044 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:06:12.947090 (+ 46us) client_negotiation.cc:769] Sending connection context
0923 15:06:12.947161 (+ 71us) client_negotiation.cc:240] Negotiation successful
0923 15:06:12.947436 (+ 275us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":33,"mutex_wait_us":118}
I20260923 15:06:12.948262 3903 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.75.188" port: 34245 } has no permanent_uuid. Determining permanent_uuid...
I20260923 15:06:12.948644 3905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:12.948522 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.188:34245 (local address 127.0.0.1:52710)
0923 15:06:12.948573 (+ 51us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:12.948623 (+ 50us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.188:34245: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":34}
W20260923 15:06:12.948796 3853 proxy.cc:239] Call had error, refreshing address and retrying: Network error: Client connection negotiation failed: client connection to 127.13.75.188:34245: connect: Connection refused (error 111)
I20260923 15:06:12.949218 3905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:12.949122 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.188:34245 (local address 127.0.0.1:52714)
0923 15:06:12.949170 (+ 48us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:12.949195 (+ 25us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.188:34245: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":32}
W20260923 15:06:12.949322 3903 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.75.188:34245: Network error: Client connection negotiation failed: client connection to 127.13.75.188:34245: connect: Connection refused (error 111)
Authenticating as principal test-admin/admin@KRBTEST.COM with password.
Entry for principal kudu/127.13.75.188 with kvno 2, encryption type aes256-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.188.keytab.
Entry for principal kudu/127.13.75.188 with kvno 2, encryption type aes128-cts-hmac-sha1-96 added to keytab WRFILE:/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.188.keytab.
I20260923 15:06:12.962007 13614 external_mini_cluster.cc:1326] Running /tmp/dist-test-taskKC4soy/build/release/bin/kudu
/tmp/dist-test-taskKC4soy/build/release/bin/kudu
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790175691491353-13614-0/minicluster-data/master-2/wal
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790175691491353-13614-0/minicluster-data/master-2/data
--block_manager=log
--webserver_interface=localhost
--never_fsync
--enable_minidumps=false
--redact=none
--metrics_log_interval_ms=1000
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790175691491353-13614-0/minicluster-data/master-2/logs
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790175691491353-13614-0/minicluster-data/master-2/data/info.pb
--server_dump_info_format=pb
--rpc_server_allow_ephemeral_ports
--unlock_experimental_flags
--unlock_unsafe_flags
--logtostderr
--logbuflevel=-1
--ipki_server_key_size=768
--openssl_security_level_override=0
master
run
--ipki_ca_key_size=768
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.75.188:34245
--webserver_interface=127.13.75.188
--webserver_port=0
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.188.keytab
--principal=kudu/127.13.75.188
--rpc_authentication=required
--superuser_acl=test-admin
--user_acl=test-user
--builtin_ntp_servers=127.13.75.148:42955
--builtin_ntp_poll_interval_ms=100
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--rpc_reuseport=true
--master_addresses=127.13.75.190:42059,127.13.75.189:39767,127.13.75.188:34245
--encrypt_data_at_rest=true
--rpc_trace_negotiation
--enable_leader_failure_detection=false
--rpc_authentication=required
--rpc_encryption=required with env {KRB5CCNAME=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790175691491353-13614-0/krb5kdc/krb5cc,KRB5_CONFIG=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790175691491353-13614-0/krb5kdc/krb5.conf,KRB5_KDC_PROFILE=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790175691491353-13614-0/krb5kdc/kdc.conf,KUDU_ENABLE_KRB5_REALM_FIX=yes}
I20260923 15:06:12.996251 3828 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.75.189" port: 39767 } attempt: 3
I20260923 15:06:12.998574 3903 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.75.188" port: 34245 } attempt: 1
I20260923 15:06:12.999194 3905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:12.999030 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.188:34245 (local address 127.0.0.1:52730)
0923 15:06:12.999097 (+ 67us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:12.999137 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.188:34245: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
I20260923 15:06:12.999887 3905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:12.999744 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.188:34245 (local address 127.0.0.1:52742)
0923 15:06:12.999808 (+ 64us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:12.999850 (+ 42us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.188:34245: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":42}
W20260923 15:06:13.000290 3903 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.75.188:34245: Network error: Client connection negotiation failed: client connection to 127.13.75.188:34245: connect: Connection refused (error 111)
Sep 23 15:06:12 dist-test-slave-cnb0 krb5kdc[3747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175972, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.190@KRBTEST.COM for kudu/127.13.75.189@KRBTEST.COM
I20260923 15:06:13.006045 3831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:12.996687 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.189:39767 (local address 127.0.0.1:48328)
0923 15:06:12.996772 (+ 85us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:12.996790 (+ 18us) client_negotiation.cc:174] Beginning negotiation
0923 15:06:12.996867 (+ 77us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:06:12.997044 (+ 177us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:06:12.997045 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:06:12.997265 (+ 220us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:06:12.997350 (+ 85us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:12.997352 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:12.998091 (+ 739us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:12.998092 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:12.999082 (+ 990us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:12.999087 (+ 5us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:12.999902 (+ 815us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:12.999904 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:13.000027 (+ 123us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:13.000588 (+ 561us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:06:13.000603 (+ 15us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:06:13.002198 (+ 1595us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:06:13.004589 (+ 2391us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:06:13.004590 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:06:13.004590 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:06:13.004783 (+ 193us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:06:13.005183 (+ 400us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:06:13.005185 (+ 2us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:06:13.005185 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:06:13.005260 (+ 75us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:06:13.005707 (+ 447us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:06:13.005709 (+ 2us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:06:13.005781 (+ 72us) client_negotiation.cc:769] Sending connection context
0923 15:06:13.005857 (+ 76us) client_negotiation.cc:240] Negotiation successful
0923 15:06:13.005961 (+ 104us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":50}
I20260923 15:06:13.006098 3907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:12.996784 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48328 (local address 127.13.75.189:39767)
0923 15:06:12.996846 (+ 62us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:12.996848 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:12.996866 (+ 18us) server_negotiation.cc:407] Connection header received
0923 15:06:12.996954 (+ 88us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:12.996955 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:12.996967 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:12.997015 (+ 48us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:12.997490 (+ 475us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:12.997991 (+ 501us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:12.999187 (+ 1196us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:12.999729 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:12.999872 (+ 143us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:13.002301 (+ 2429us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:06:13.002303 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:06:13.002303 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:06:13.002312 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:06:13.004348 (+ 2036us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:13.004881 (+ 533us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:13.004882 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:13.004883 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:13.004991 (+ 108us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:13.005380 (+ 389us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:13.005381 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:13.005382 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:13.005588 (+ 206us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:06:13.005675 (+ 87us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:13.005900 (+ 225us) server_negotiation.cc:299] Negotiation successful
0923 15:06:13.005992 (+ 92us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":35}
I20260923 15:06:13.006779 3828 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.75.188" port: 34245 } has no permanent_uuid. Determining permanent_uuid...
I20260923 15:06:13.007218 3831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:13.007096 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.188:34245 (local address 127.0.0.1:52748)
0923 15:06:13.007151 (+ 55us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:13.007190 (+ 39us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.188:34245: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":35}
I20260923 15:06:13.007567 3831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:13.007481 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.188:34245 (local address 127.0.0.1:52760)
0923 15:06:13.007523 (+ 42us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:13.007549 (+ 26us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.188:34245: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":29}
W20260923 15:06:13.007743 3828 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.75.188:34245: Network error: Client connection negotiation failed: client connection to 127.13.75.188:34245: connect: Connection refused (error 111)
I20260923 15:06:13.038966 3828 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.75.188" port: 34245 } attempt: 1
I20260923 15:06:13.039585 3831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:13.039415 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.188:34245 (local address 127.0.0.1:52776)
0923 15:06:13.039506 (+ 91us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:13.039551 (+ 45us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.188:34245: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":63}
I20260923 15:06:13.040099 3831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:13.039961 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.188:34245 (local address 127.0.0.1:52790)
0923 15:06:13.040032 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:13.040072 (+ 40us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.188:34245: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":49}
W20260923 15:06:13.040270 3828 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.75.188:34245: Network error: Client connection negotiation failed: client connection to 127.13.75.188:34245: connect: Connection refused (error 111)
W20260923 15:06:13.052661 3910 flags.cc:425] Enabled unsafe flag: --enable_leader_failure_detection=false
W20260923 15:06:13.052918 3910 flags.cc:425] Enabled unsafe flag: --openssl_security_level_override=0
W20260923 15:06:13.052939 3910 flags.cc:425] Enabled unsafe flag: --rpc_server_allow_ephemeral_ports=true
W20260923 15:06:13.052965 3910 flags.cc:425] Enabled unsafe flag: --never_fsync=true
W20260923 15:06:13.054270 3910 flags.cc:425] Enabled experimental flag: --ipki_ca_key_size=768
W20260923 15:06:13.054313 3910 flags.cc:425] Enabled experimental flag: --rpc_trace_negotiation=true
W20260923 15:06:13.054329 3910 flags.cc:425] Enabled experimental flag: --ipki_server_key_size=768
W20260923 15:06:13.054343 3910 flags.cc:425] Enabled experimental flag: --tsk_num_rsa_bits=512
W20260923 15:06:13.054358 3910 flags.cc:425] Enabled experimental flag: --rpc_reuseport=true
I20260923 15:06:13.056113 3910 master_runner.cc:386] Master server non-default flags:
--builtin_ntp_poll_interval_ms=100
--builtin_ntp_servers=127.13.75.148:42955
--ntp_initial_sync_wait_secs=10
--time_source=builtin
--enable_leader_failure_detection=false
--fs_data_dirs=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790175691491353-13614-0/minicluster-data/master-2/data
--fs_wal_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790175691491353-13614-0/minicluster-data/master-2/wal
--ipki_ca_key_size=768
--master_addresses=127.13.75.190:42059,127.13.75.189:39767,127.13.75.188:34245
--rpc_trace_negotiation=true
--keytab_file=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790175691491353-13614-0/krb5kdc/kudu_127.13.75.188.keytab
--principal=kudu/127.13.75.188
--ipki_server_key_size=768
--openssl_security_level_override=0
--tsk_num_rsa_bits=512
--rpc_bind_addresses=127.13.75.188:34245
--rpc_reuseport=true
--rpc_server_allow_ephemeral_ports=true
--rpc_authentication=required
--rpc_encryption=required
--superuser_acl=<redacted>
--user_acl=<redacted>
--metrics_log_interval_ms=1000
--server_dump_info_format=pb
--server_dump_info_path=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790175691491353-13614-0/minicluster-data/master-2/data/info.pb
--webserver_interface=127.13.75.188
--webserver_port=0
--encrypt_data_at_rest=true
--never_fsync=true
--heap_profile_path=/tmp/kudu.3910
--redact=none
--unlock_experimental_flags=true
--unlock_unsafe_flags=true
--enable_minidumps=false
--log_dir=/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790175691491353-13614-0/minicluster-data/master-2/logs
--logbuflevel=-1
--logtostderr=true
Master server version:
kudu 1.18.0-SNAPSHOT
revision c2d25d5f57d102d9ad9869eaf208d035a22e18aa
build type RELEASE
built by systest at 23 Sep 2026 14:06:54 UTC on ccycloud.quasar-athtfy.root.comops.site
build id 4667496
I20260923 15:06:13.056363 3910 env_posix.cc:2256] Not raising this process' open files per process limit of 1048576; it is already as high as it can go
I20260923 15:06:13.056639 3910 file_cache.cc:493] Constructed file cache file cache with capacity 419430
W20260923 15:06:13.062292 3918 instance_detector.cc:116] could not retrieve OpenStack instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
W20260923 15:06:13.062276 3915 instance_detector.cc:116] could not retrieve AWS instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:06:13.062314 3910 server_base.cc:1034] running on GCE node
W20260923 15:06:13.062281 3916 instance_detector.cc:116] could not retrieve Azure instance metadata: Network error: curl error: HTTP response code said error: The requested URL returned error: 404
I20260923 15:06:13.062840 3910 hybrid_clock.cc:584] initializing the hybrid clock with 'builtin' time source
I20260923 15:06:13.063221 3910 hybrid_clock.cc:630] waiting up to --ntp_initial_sync_wait_secs=10 seconds for the clock to synchronize
I20260923 15:06:13.064391 3910 hybrid_clock.cc:648] HybridClock initialized: now 1790175973064357 us; error 51 us; skew 500 ppm
Sep 23 15:06:13 dist-test-slave-cnb0 krb5kdc[3747](info): AS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175973, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.188@KRBTEST.COM for krbtgt/KRBTEST.COM@KRBTEST.COM
I20260923 15:06:13.067181 3910 init.cc:377] Logged in from keytab as kudu/127.13.75.188@KRBTEST.COM (short username kudu)
I20260923 15:06:13.067744 3910 webserver.cc:460] Webserver started at http://127.13.75.188:38495/ using document root <none> and password file <none>
I20260923 15:06:13.068020 3910 fs_manager.cc:359] Metadata directory not provided
I20260923 15:06:13.068092 3910 fs_manager.cc:365] Using write-ahead log directory (fs_wal_dir) as metadata directory
I20260923 15:06:13.068250 3910 server_base.cc:882] This appears to be a new deployment of Kudu; creating new FS layout
I20260923 15:06:13.068471 3903 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.75.188" port: 34245 } attempt: 2
I20260923 15:06:13.068933 3905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:13.068778 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.188:34245 (local address 127.0.0.1:52796)
0923 15:06:13.068849 (+ 71us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:13.068892 (+ 43us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.188:34245: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":40}
I20260923 15:06:13.069339 3905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:13.069212 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.188:34245 (local address 127.0.0.1:52810)
0923 15:06:13.069284 (+ 72us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:13.069315 (+ 31us) negotiation.cc:327] Negotiation complete: Network error: Client connection negotiation failed: client connection to 127.13.75.188:34245: connect: Connection refused (error 111)
Metrics: {"client-negotiator.queue_time_us":54}
W20260923 15:06:13.069497 3903 consensus_peers.cc:646] Error getting permanent uuid from config peer 127.13.75.188:34245: Network error: Client connection negotiation failed: client connection to 127.13.75.188:34245: connect: Connection refused (error 111)
I20260923 15:06:13.069729 3910 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790175691491353-13614-0/minicluster-data/master-2/data/instance:
uuid: "617b9a2ed1c44802be824f496c8144b6"
format_stamp: "Formatted at 2026-09-23 15:06:13 on dist-test-slave-cnb0"
server_key: "a8b3449778eb2d828a5334d715da83f8"
server_key_iv: "f88bf9f6eaa28625bcdfc92c6bcca640"
server_key_version: "encryptionkey@0"
I20260923 15:06:13.070072 3910 fs_manager.cc:1063] Generated new instance metadata in path /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790175691491353-13614-0/minicluster-data/master-2/wal/instance:
uuid: "617b9a2ed1c44802be824f496c8144b6"
format_stamp: "Formatted at 2026-09-23 15:06:13 on dist-test-slave-cnb0"
server_key: "a8b3449778eb2d828a5334d715da83f8"
server_key_iv: "f88bf9f6eaa28625bcdfc92c6bcca640"
server_key_version: "encryptionkey@0"
I20260923 15:06:13.071611 3910 fs_manager.cc:691] Time spent creating directory manager: real 0.001s user 0.004s sys 0.000s
I20260923 15:06:13.072412 3925 log_block_manager.cc:3774] Time spent loading block containers with low live blocks: real 0.000s user 0.000s sys 0.000s
I20260923 15:06:13.072585 3910 fs_manager.cc:725] Time spent opening block manager: real 0.001s user 0.001s sys 0.000s
I20260923 15:06:13.072724 3910 fs_manager.cc:642] Opened local filesystem: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790175691491353-13614-0/minicluster-data/master-2/data,/tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790175691491353-13614-0/minicluster-data/master-2/wal
uuid: "617b9a2ed1c44802be824f496c8144b6"
format_stamp: "Formatted at 2026-09-23 15:06:13 on dist-test-slave-cnb0"
server_key: "a8b3449778eb2d828a5334d715da83f8"
server_key_iv: "f88bf9f6eaa28625bcdfc92c6bcca640"
server_key_version: "encryptionkey@0"
I20260923 15:06:13.072811 3910 fs_report.cc:383] FS layout report
--------------------
wal directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790175691491353-13614-0/minicluster-data/master-2/wal
metadata directory: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790175691491353-13614-0/minicluster-data/master-2/wal
1 data directories: /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790175691491353-13614-0/minicluster-data/master-2/data/data
Total live blocks: 0
Total live bytes: 0
Total live bytes (after alignment): 0
Total number of LBM containers: 0 (0 full)
Did not check for missing blocks
Did not check for orphaned blocks
Total full LBM containers with extra space: 0 (0 repaired)
Total full LBM container extra space in bytes: 0 (0 repaired)
Total incomplete LBM containers: 0 (0 repaired)
Total LBM partial records: 0 (0 repaired)
Total corrupted LBM metadata records in RocksDB: 0 (0 repaired)
I20260923 15:06:13.083736 3910 rpc_server.cc:225] running with OpenSSL 1.0.2k-fips 26 Jan 2017
I20260923 15:06:13.086052 3910 env_posix.cc:2256] Not raising this process' running threads per effective uid limit of 18446744073709551615; it is already as high as it can go
I20260923 15:06:13.086248 3910 kserver.cc:163] Server-wide thread pool size limit: 3276
I20260923 15:06:13.090644 3910 rpc_server.cc:307] RPC server started. Bound to: 127.13.75.188:34245
I20260923 15:06:13.090770 3978 acceptor_pool.cc:254] collecting diagnostics on the listening RPC socket 127.13.75.188:34245 every 8 connection(s)
I20260923 15:06:13.091280 3910 server_base.cc:1166] Dumped server information to /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790175691491353-13614-0/minicluster-data/master-2/data/info.pb
I20260923 15:06:13.092360 3980 data_dirs.cc:395] Could only allocate 1 dirs of requested 3 for tablet 00000000000000000000000000000000. 1 dirs total, 0 dirs full, 0 dirs failed
I20260923 15:06:13.094059 3980 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.75.190" port: 42059 } has no permanent_uuid. Determining permanent_uuid...
I20260923 15:06:13.097266 13614 external_mini_cluster.cc:1388] Started /tmp/dist-test-taskKC4soy/build/release/bin/kudu as pid 3910
I20260923 15:06:13.097388 13614 external_mini_cluster.cc:1402] Reading /tmp/dist-test-taskKC4soy/test-tmp/security-itest.0.ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys_1.1790175691491353-13614-0/minicluster-data/master-2/wal/instance
I20260923 15:06:13.097620 13614 external_mini_cluster.cc:1428] Setting key 82996ebd52c107a8a0791efd3ff0a9d2
Sep 23 15:06:13 dist-test-slave-cnb0 krb5kdc[3747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175973, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.188@KRBTEST.COM for kudu/127.13.75.190@KRBTEST.COM
Sep 23 15:06:13 dist-test-slave-cnb0 krb5kdc[3747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175972, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.75.188@KRBTEST.COM
I20260923 15:06:13.105014 3982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:13.094817 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.190:42059 (local address 127.0.0.1:52024)
0923 15:06:13.095255 (+ 438us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:13.095303 (+ 48us) client_negotiation.cc:174] Beginning negotiation
0923 15:06:13.095805 (+ 502us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:06:13.096078 (+ 273us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:06:13.096082 (+ 4us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:06:13.096349 (+ 267us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:06:13.096554 (+ 205us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:13.096558 (+ 4us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:13.097274 (+ 716us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:13.097276 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:13.098549 (+ 1273us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:13.098551 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:13.099235 (+ 684us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:13.099237 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:13.099385 (+ 148us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:13.099980 (+ 595us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:06:13.099991 (+ 11us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:06:13.101490 (+ 1499us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:06:13.103240 (+ 1750us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:06:13.103248 (+ 8us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:06:13.103250 (+ 2us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:06:13.103483 (+ 233us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:06:13.103735 (+ 252us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:06:13.103736 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:06:13.103736 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:06:13.103854 (+ 118us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:06:13.104207 (+ 353us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:06:13.104212 (+ 5us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:06:13.104390 (+ 178us) client_negotiation.cc:769] Sending connection context
0923 15:06:13.104532 (+ 142us) client_negotiation.cc:240] Negotiation successful
0923 15:06:13.104719 (+ 187us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":381,"thread_start_us":142,"threads_started":1}
I20260923 15:06:13.105115 3832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:13.094973 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52024 (local address 127.13.75.190:42059)
0923 15:06:13.095045 (+ 72us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:13.095048 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:13.095419 (+ 371us) server_negotiation.cc:407] Connection header received
0923 15:06:13.095928 (+ 509us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:13.095929 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:13.095940 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:13.096003 (+ 63us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:13.096666 (+ 663us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:13.097160 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:13.098654 (+ 1494us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:13.099062 (+ 408us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:13.099152 (+ 90us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:13.101583 (+ 2431us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:06:13.101585 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:06:13.101585 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:06:13.101599 (+ 14us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:06:13.103123 (+ 1524us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:13.103559 (+ 436us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:13.103561 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:13.103562 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:13.103627 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:13.103909 (+ 282us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:13.103911 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:13.103911 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:13.104066 (+ 155us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:06:13.104142 (+ 76us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:13.104925 (+ 783us) server_negotiation.cc:299] Negotiation successful
0923 15:06:13.105018 (+ 93us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":43}
I20260923 15:06:13.105623 3980 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.75.189" port: 39767 } has no permanent_uuid. Determining permanent_uuid...
I20260923 15:06:13.108029 3984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:13.098059 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52820 (local address 127.13.75.188:34245)
0923 15:06:13.098254 (+ 195us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:13.098259 (+ 5us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:13.098304 (+ 45us) server_negotiation.cc:407] Connection header received
0923 15:06:13.098359 (+ 55us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:13.098362 (+ 3us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:13.098383 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:13.098501 (+ 118us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:13.099402 (+ 901us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:13.100435 (+ 1033us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:13.101186 (+ 751us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:13.101544 (+ 358us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:13.101636 (+ 92us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:13.104196 (+ 2560us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:06:13.104204 (+ 8us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:06:13.104204 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:06:13.104211 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:06:13.106376 (+ 2165us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:13.106971 (+ 595us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:13.106975 (+ 4us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:13.106975 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:13.107054 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:13.107319 (+ 265us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:13.107320 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:13.107320 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:13.107572 (+ 252us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:06:13.107750 (+ 178us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:13.107829 (+ 79us) server_negotiation.cc:299] Negotiation successful
0923 15:06:13.107914 (+ 85us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":153,"thread_start_us":102,"threads_started":1}
Sep 23 15:06:13 dist-test-slave-cnb0 krb5kdc[3747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175973, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.188@KRBTEST.COM for kudu/127.13.75.189@KRBTEST.COM
I20260923 15:06:13.113495 3907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:13.105961 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48334 (local address 127.13.75.189:39767)
0923 15:06:13.106007 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:13.106009 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:13.106057 (+ 48us) server_negotiation.cc:407] Connection header received
0923 15:06:13.106140 (+ 83us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:13.106140 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:13.106150 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:13.106191 (+ 41us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:13.106546 (+ 355us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:13.107378 (+ 832us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:13.108354 (+ 976us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:13.108584 (+ 230us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:13.108662 (+ 78us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:13.110430 (+ 1768us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:06:13.110432 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:06:13.110432 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:06:13.110440 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:06:13.112232 (+ 1792us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:13.112600 (+ 368us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:13.112602 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:13.112602 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:13.112672 (+ 70us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:13.112904 (+ 232us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:13.112905 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:13.112906 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:13.113062 (+ 156us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:06:13.113123 (+ 61us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:13.113354 (+ 231us) server_negotiation.cc:299] Negotiation successful
0923 15:06:13.113422 (+ 68us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":20}
I20260923 15:06:13.113529 3982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:13.105889 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.189:39767 (local address 127.0.0.1:48334)
0923 15:06:13.106005 (+ 116us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:13.106017 (+ 12us) client_negotiation.cc:174] Beginning negotiation
0923 15:06:13.106074 (+ 57us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:06:13.106215 (+ 141us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:06:13.106216 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:06:13.106406 (+ 190us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:06:13.106456 (+ 50us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:13.106458 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:13.107458 (+ 1000us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:13.107459 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:13.108279 (+ 820us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:13.108281 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:13.108658 (+ 377us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:13.108659 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:13.108720 (+ 61us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:13.109167 (+ 447us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:06:13.109171 (+ 4us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:06:13.110349 (+ 1178us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:06:13.112339 (+ 1990us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:06:13.112340 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:06:13.112340 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:06:13.112518 (+ 178us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:06:13.112766 (+ 248us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:06:13.112766 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:06:13.112767 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:06:13.112827 (+ 60us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:06:13.113145 (+ 318us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:06:13.113146 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:06:13.113236 (+ 90us) client_negotiation.cc:769] Sending connection context
0923 15:06:13.113322 (+ 86us) client_negotiation.cc:240] Negotiation successful
0923 15:06:13.113428 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":98}
Sep 23 15:06:13 dist-test-slave-cnb0 krb5kdc[3747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175972, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.75.190@KRBTEST.COM
I20260923 15:06:13.113977 3980 sys_catalog.cc:422] member_type: VOTER last_known_addr { host: "127.13.75.188" port: 34245 } has no permanent_uuid. Determining permanent_uuid...
I20260923 15:06:13.115435 3828 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.75.188" port: 34245 } attempt: 2
I20260923 15:06:13.117165 3832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:13.108887 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52040 (local address 127.13.75.190:42059)
0923 15:06:13.108946 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:13.108948 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:13.108956 (+ 8us) server_negotiation.cc:407] Connection header received
0923 15:06:13.108979 (+ 23us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:13.108980 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:13.108990 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:13.109071 (+ 81us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:13.109557 (+ 486us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:13.110051 (+ 494us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:13.111025 (+ 974us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:13.111292 (+ 267us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:13.111366 (+ 74us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:13.114111 (+ 2745us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:06:13.114112 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:06:13.114113 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:06:13.114120 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:06:13.115835 (+ 1715us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:13.116296 (+ 461us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:13.116298 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:13.116298 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:13.116365 (+ 67us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:13.116590 (+ 225us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:13.116591 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:13.116591 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:13.116740 (+ 149us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:06:13.116920 (+ 180us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:13.116983 (+ 63us) server_negotiation.cc:299] Negotiation successful
0923 15:06:13.117069 (+ 86us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":33}
Sep 23 15:06:13 dist-test-slave-cnb0 krb5kdc[3747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175973, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.188@KRBTEST.COM for kudu/127.13.75.188@KRBTEST.COM
Sep 23 15:06:13 dist-test-slave-cnb0 krb5kdc[3747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175972, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.190@KRBTEST.COM for kudu/127.13.75.188@KRBTEST.COM
I20260923 15:06:13.122318 3982 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:13.114244 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.188:34245 (local address 127.0.0.1:52826)
0923 15:06:13.114376 (+ 132us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:13.114386 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 15:06:13.114429 (+ 43us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:06:13.114535 (+ 106us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:06:13.114536 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:06:13.114965 (+ 429us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:06:13.115012 (+ 47us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:13.115014 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:13.115744 (+ 730us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:13.115745 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:13.116370 (+ 625us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:13.116372 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:13.116920 (+ 548us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:13.116921 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:13.116978 (+ 57us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:13.117519 (+ 541us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:06:13.117525 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:06:13.119244 (+ 1719us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:06:13.121072 (+ 1828us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:06:13.121073 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:06:13.121074 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:06:13.121393 (+ 319us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:06:13.121636 (+ 243us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:06:13.121636 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:06:13.121637 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:06:13.121702 (+ 65us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:06:13.121969 (+ 267us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:06:13.121970 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:06:13.122005 (+ 35us) client_negotiation.cc:769] Sending connection context
0923 15:06:13.122073 (+ 68us) client_negotiation.cc:240] Negotiation successful
0923 15:06:13.122179 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":119}
I20260923 15:06:13.122421 3984 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:13.114291 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52826 (local address 127.13.75.188:34245)
0923 15:06:13.114336 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:13.114338 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:13.114434 (+ 96us) server_negotiation.cc:407] Connection header received
0923 15:06:13.114485 (+ 51us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:13.114486 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:13.114497 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:13.114535 (+ 38us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:13.115092 (+ 557us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:13.115653 (+ 561us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:13.116454 (+ 801us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:13.116801 (+ 347us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:13.116871 (+ 70us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:13.119317 (+ 2446us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:06:13.119318 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:06:13.119319 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:06:13.119325 (+ 6us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:06:13.120978 (+ 1653us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:13.121490 (+ 512us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:13.121491 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:13.121492 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:13.121556 (+ 64us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:13.121768 (+ 212us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:13.121769 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:13.121769 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:13.121880 (+ 111us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:06:13.121949 (+ 69us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:13.122144 (+ 195us) server_negotiation.cc:299] Negotiation successful
0923 15:06:13.122266 (+ 122us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"mutex_wait_us":3,"server-negotiator.queue_time_us":24}
I20260923 15:06:13.124290 3831 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:13.115671 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.188:34245 (local address 127.0.0.1:52828)
0923 15:06:13.115720 (+ 49us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:13.115730 (+ 10us) client_negotiation.cc:174] Beginning negotiation
0923 15:06:13.115764 (+ 34us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:06:13.116035 (+ 271us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:06:13.116036 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:06:13.116212 (+ 176us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:06:13.116267 (+ 55us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:13.116269 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:13.117114 (+ 845us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:13.117115 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:13.117839 (+ 724us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:13.117841 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:13.118292 (+ 451us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:13.118293 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:13.118366 (+ 73us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:13.118705 (+ 339us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:06:13.118711 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:06:13.120323 (+ 1612us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:06:13.123085 (+ 2762us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:06:13.123086 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:06:13.123087 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:06:13.123292 (+ 205us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:06:13.123499 (+ 207us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:06:13.123499 (+ 0us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:06:13.123500 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:06:13.123572 (+ 72us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:06:13.123923 (+ 351us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:06:13.123924 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:06:13.123980 (+ 56us) client_negotiation.cc:769] Sending connection context
0923 15:06:13.124059 (+ 79us) client_negotiation.cc:240] Negotiation successful
0923 15:06:13.124168 (+ 109us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":32}
Sep 23 15:06:13 dist-test-slave-cnb0 krb5kdc[3747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175972, etypes {rep=17 tkt=17 ses=17}, test-admin@KRBTEST.COM for kudu/127.13.75.189@KRBTEST.COM
I20260923 15:06:13.124796 3985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:13.115730 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52828 (local address 127.13.75.188:34245)
0923 15:06:13.115909 (+ 179us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:13.115911 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:13.115918 (+ 7us) server_negotiation.cc:407] Connection header received
0923 15:06:13.115934 (+ 16us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:13.115935 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:13.115944 (+ 9us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:13.116012 (+ 68us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:13.116343 (+ 331us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:13.117022 (+ 679us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:13.117937 (+ 915us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:13.118236 (+ 299us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:13.118286 (+ 50us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:13.120428 (+ 2142us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:06:13.120430 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:06:13.120430 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:06:13.120437 (+ 7us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:06:13.122976 (+ 2539us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:13.123366 (+ 390us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:13.123367 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:13.123367 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:13.123432 (+ 65us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:13.123643 (+ 211us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:13.123644 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:13.123644 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:13.123839 (+ 195us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:06:13.123910 (+ 71us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:13.124434 (+ 524us) server_negotiation.cc:299] Negotiation successful
0923 15:06:13.124540 (+ 106us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":159,"thread_start_us":108,"threads_started":1}
I20260923 15:06:13.124956 3980 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 617b9a2ed1c44802be824f496c8144b6: Bootstrap starting.
I20260923 15:06:13.125756 3980 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 617b9a2ed1c44802be824f496c8144b6: Neither blocks nor log segments found. Creating new log.
I20260923 15:06:13.126088 3980 log.cc:824] T 00000000000000000000000000000000 P 617b9a2ed1c44802be824f496c8144b6: Log is configured to *not* fsync() on all Append() calls
I20260923 15:06:13.126394 3828 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 925e4dec28a444a0b61b5bcb77464ce1: Bootstrap starting.
I20260923 15:06:13.127161 3828 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 925e4dec28a444a0b61b5bcb77464ce1: Neither blocks nor log segments found. Creating new log.
I20260923 15:06:13.127197 3980 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 617b9a2ed1c44802be824f496c8144b6: No bootstrap required, opened a new log
I20260923 15:06:13.127498 3828 log.cc:824] T 00000000000000000000000000000000 P 925e4dec28a444a0b61b5bcb77464ce1: Log is configured to *not* fsync() on all Append() calls
I20260923 15:06:13.128432 3828 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 925e4dec28a444a0b61b5bcb77464ce1: No bootstrap required, opened a new log
I20260923 15:06:13.128628 3907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:13.118188 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48336 (local address 127.13.75.189:39767)
0923 15:06:13.118240 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:13.118242 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:13.118962 (+ 720us) server_negotiation.cc:407] Connection header received
0923 15:06:13.119042 (+ 80us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:13.119043 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:13.119056 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:13.119115 (+ 59us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:13.119810 (+ 695us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:13.120579 (+ 769us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:13.121556 (+ 977us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:13.121857 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:13.121938 (+ 81us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:13.125284 (+ 3346us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:06:13.125287 (+ 3us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:06:13.125287 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:06:13.125297 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:06:13.127225 (+ 1928us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:13.127699 (+ 474us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:13.127700 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:13.127701 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:13.127780 (+ 79us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:13.128043 (+ 263us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:13.128044 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:13.128044 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:13.128190 (+ 146us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:06:13.128390 (+ 200us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:13.128431 (+ 41us) server_negotiation.cc:299] Negotiation successful
0923 15:06:13.128514 (+ 83us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":29}
I20260923 15:06:13.129556 3980 raft_consensus.cc:348] T 00000000000000000000000000000000 P 617b9a2ed1c44802be824f496c8144b6 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "925e4dec28a444a0b61b5bcb77464ce1" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 42059 } } peers { permanent_uuid: "33e89a99a9f146cb93c7445c741be846" member_type: VOTER last_known_addr { host: "127.13.75.189" port: 39767 } } peers { permanent_uuid: "617b9a2ed1c44802be824f496c8144b6" member_type: VOTER last_known_addr { host: "127.13.75.188" port: 34245 } }
I20260923 15:06:13.129719 3980 raft_consensus.cc:729] T 00000000000000000000000000000000 P 617b9a2ed1c44802be824f496c8144b6 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 617b9a2ed1c44802be824f496c8144b6, State: Initialized, Role: FOLLOWER
I20260923 15:06:13.129812 3980 consensus_queue.cc:260] T 00000000000000000000000000000000 P 617b9a2ed1c44802be824f496c8144b6 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "925e4dec28a444a0b61b5bcb77464ce1" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 42059 } } peers { permanent_uuid: "33e89a99a9f146cb93c7445c741be846" member_type: VOTER last_known_addr { host: "127.13.75.189" port: 39767 } } peers { permanent_uuid: "617b9a2ed1c44802be824f496c8144b6" member_type: VOTER last_known_addr { host: "127.13.75.188" port: 34245 } }
I20260923 15:06:13.130138 13614 external_mini_cluster.cc:930] 0 TS(s) registered with all masters
I20260923 15:06:13.130270 3980 sys_catalog.cc:564] T 00000000000000000000000000000000 P 617b9a2ed1c44802be824f496c8144b6 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 15:06:13.130596 3828 raft_consensus.cc:348] T 00000000000000000000000000000000 P 925e4dec28a444a0b61b5bcb77464ce1 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "925e4dec28a444a0b61b5bcb77464ce1" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 42059 } } peers { permanent_uuid: "33e89a99a9f146cb93c7445c741be846" member_type: VOTER last_known_addr { host: "127.13.75.189" port: 39767 } } peers { permanent_uuid: "617b9a2ed1c44802be824f496c8144b6" member_type: VOTER last_known_addr { host: "127.13.75.188" port: 34245 } }
I20260923 15:06:13.130800 3828 raft_consensus.cc:729] T 00000000000000000000000000000000 P 925e4dec28a444a0b61b5bcb77464ce1 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 925e4dec28a444a0b61b5bcb77464ce1, State: Initialized, Role: FOLLOWER
I20260923 15:06:13.130941 3828 consensus_queue.cc:260] T 00000000000000000000000000000000 P 925e4dec28a444a0b61b5bcb77464ce1 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "925e4dec28a444a0b61b5bcb77464ce1" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 42059 } } peers { permanent_uuid: "33e89a99a9f146cb93c7445c741be846" member_type: VOTER last_known_addr { host: "127.13.75.189" port: 39767 } } peers { permanent_uuid: "617b9a2ed1c44802be824f496c8144b6" member_type: VOTER last_known_addr { host: "127.13.75.188" port: 34245 } }
I20260923 15:06:13.131135 3802 tablet_service.cc:1939] Received Run Leader Election RPC: tablet_id: "00000000000000000000000000000000"
dest_uuid: "925e4dec28a444a0b61b5bcb77464ce1"
from {username='test-admin', principal='test-admin@KRBTEST.COM'} at 127.0.0.1:52040
I20260923 15:06:13.131237 3994 sys_catalog.cc:455] T 00000000000000000000000000000000 P 925e4dec28a444a0b61b5bcb77464ce1 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "925e4dec28a444a0b61b5bcb77464ce1" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 42059 } } peers { permanent_uuid: "33e89a99a9f146cb93c7445c741be846" member_type: VOTER last_known_addr { host: "127.13.75.189" port: 39767 } } peers { permanent_uuid: "617b9a2ed1c44802be824f496c8144b6" member_type: VOTER last_known_addr { host: "127.13.75.188" port: 34245 } } }
I20260923 15:06:13.131318 3828 sys_catalog.cc:564] T 00000000000000000000000000000000 P 925e4dec28a444a0b61b5bcb77464ce1 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 15:06:13.131378 3802 raft_consensus.cc:482] T 00000000000000000000000000000000 P 925e4dec28a444a0b61b5bcb77464ce1 [term 0 FOLLOWER]: Starting forced leader election (received explicit request)
I20260923 15:06:13.131448 3802 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 925e4dec28a444a0b61b5bcb77464ce1 [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:06:13.131870 3988 sys_catalog.cc:455] T 00000000000000000000000000000000 P 617b9a2ed1c44802be824f496c8144b6 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "925e4dec28a444a0b61b5bcb77464ce1" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 42059 } } peers { permanent_uuid: "33e89a99a9f146cb93c7445c741be846" member_type: VOTER last_known_addr { host: "127.13.75.189" port: 39767 } } peers { permanent_uuid: "617b9a2ed1c44802be824f496c8144b6" member_type: VOTER last_known_addr { host: "127.13.75.188" port: 34245 } } }
I20260923 15:06:13.131982 3988 sys_catalog.cc:458] T 00000000000000000000000000000000 P 617b9a2ed1c44802be824f496c8144b6 [sys.catalog]: This master's current role is: FOLLOWER
I20260923 15:06:13.132489 3802 raft_consensus.cc:504] T 00000000000000000000000000000000 P 925e4dec28a444a0b61b5bcb77464ce1 [term 1 FOLLOWER]: Starting forced leader election with config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "925e4dec28a444a0b61b5bcb77464ce1" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 42059 } } peers { permanent_uuid: "33e89a99a9f146cb93c7445c741be846" member_type: VOTER last_known_addr { host: "127.13.75.189" port: 39767 } } peers { permanent_uuid: "617b9a2ed1c44802be824f496c8144b6" member_type: VOTER last_known_addr { host: "127.13.75.188" port: 34245 } }
I20260923 15:06:13.132804 3802 leader_election.cc:288] T 00000000000000000000000000000000 P 925e4dec28a444a0b61b5bcb77464ce1 [CANDIDATE]: Term 1 election: Requested vote from peers 33e89a99a9f146cb93c7445c741be846 (127.13.75.189:39767), 617b9a2ed1c44802be824f496c8144b6 (127.13.75.188:34245)
I20260923 15:06:13.134064 3953 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "925e4dec28a444a0b61b5bcb77464ce1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "617b9a2ed1c44802be824f496c8144b6"
I20260923 15:06:13.134188 3953 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 617b9a2ed1c44802be824f496c8144b6 [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:06:13.135380 3953 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 617b9a2ed1c44802be824f496c8144b6 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 925e4dec28a444a0b61b5bcb77464ce1 in term 1.
I20260923 15:06:13.131317 3994 sys_catalog.cc:458] T 00000000000000000000000000000000 P 925e4dec28a444a0b61b5bcb77464ce1 [sys.catalog]: This master's current role is: FOLLOWER
I20260923 15:06:13.135671 3778 leader_election.cc:302] T 00000000000000000000000000000000 P 925e4dec28a444a0b61b5bcb77464ce1 [CANDIDATE]: Term 1 election: Election decided. Result: candidate won. Election summary: received 2 responses out of 3 voters: 2 yes votes; 0 no votes. yes voters: 617b9a2ed1c44802be824f496c8144b6, 925e4dec28a444a0b61b5bcb77464ce1; no voters:
I20260923 15:06:13.135839 3994 raft_consensus.cc:2781] T 00000000000000000000000000000000 P 925e4dec28a444a0b61b5bcb77464ce1 [term 1 FOLLOWER]: Leader election won for term 1
I20260923 15:06:13.132553 3910 master_runner.cc:186] Error getting master registration for 127.13.75.190:42059: OK, instance_id { permanent_uuid: "925e4dec28a444a0b61b5bcb77464ce1" instance_seqno: 1790175972767751 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260923 15:06:13.136072 3910 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260923 15:06:13.136052 3994 raft_consensus.cc:686] T 00000000000000000000000000000000 P 925e4dec28a444a0b61b5bcb77464ce1 [term 1 LEADER]: Becoming Leader. State: Replica: 925e4dec28a444a0b61b5bcb77464ce1, State: Running, Role: LEADER
I20260923 15:06:13.136231 3994 consensus_queue.cc:237] T 00000000000000000000000000000000 P 925e4dec28a444a0b61b5bcb77464ce1 [LEADER]: Queue going to LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 1, Majority size: 2, State: 0, Mode: LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "925e4dec28a444a0b61b5bcb77464ce1" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 42059 } } peers { permanent_uuid: "33e89a99a9f146cb93c7445c741be846" member_type: VOTER last_known_addr { host: "127.13.75.189" port: 39767 } } peers { permanent_uuid: "617b9a2ed1c44802be824f496c8144b6" member_type: VOTER last_known_addr { host: "127.13.75.188" port: 34245 } }
W20260923 15:06:13.132537 4000 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 617b9a2ed1c44802be824f496c8144b6: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260923 15:06:13.136513 4000 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260923 15:06:13.136837 3994 sys_catalog.cc:455] T 00000000000000000000000000000000 P 925e4dec28a444a0b61b5bcb77464ce1 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 925e4dec28a444a0b61b5bcb77464ce1. Latest consensus state: current_term: 1 leader_uuid: "925e4dec28a444a0b61b5bcb77464ce1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "925e4dec28a444a0b61b5bcb77464ce1" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 42059 } } peers { permanent_uuid: "33e89a99a9f146cb93c7445c741be846" member_type: VOTER last_known_addr { host: "127.13.75.189" port: 39767 } } peers { permanent_uuid: "617b9a2ed1c44802be824f496c8144b6" member_type: VOTER last_known_addr { host: "127.13.75.188" port: 34245 } } }
I20260923 15:06:13.136938 3994 sys_catalog.cc:458] T 00000000000000000000000000000000 P 925e4dec28a444a0b61b5bcb77464ce1 [sys.catalog]: This master's current role is: LEADER
W20260923 15:06:13.137346 4012 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 925e4dec28a444a0b61b5bcb77464ce1: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260923 15:06:13.137404 4012 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260923 15:06:13.134382 3760 master_runner.cc:186] Error getting master registration for 127.13.75.189:39767: OK, instance_id { permanent_uuid: "33e89a99a9f146cb93c7445c741be846" instance_seqno: 1790175972923271 } error { code: CATALOG_MANAGER_NOT_INITIALIZED status { code: SERVICE_UNAVAILABLE message: "Catalog manager is not initialized. State: Starting" } }
I20260923 15:06:13.137629 3760 master_runner.cc:418] Couldn't verify the masters in the cluster. Trying again...
I20260923 15:06:13.145486 3832 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:13.134143 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52056 (local address 127.13.75.190:42059)
0923 15:06:13.134190 (+ 47us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:13.134192 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:13.136548 (+ 2356us) server_negotiation.cc:407] Connection header received
0923 15:06:13.136689 (+ 141us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:13.136691 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:13.136704 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:13.136778 (+ 74us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:13.138038 (+ 1260us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:13.138759 (+ 721us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:13.139827 (+ 1068us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:13.140128 (+ 301us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:13.140173 (+ 45us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:13.142113 (+ 1940us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:06:13.142115 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:06:13.142115 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:06:13.142123 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:06:13.143887 (+ 1764us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:13.144332 (+ 445us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:13.144333 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:13.144334 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:13.144400 (+ 66us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:13.144618 (+ 218us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:13.144619 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:13.144619 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:13.144788 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:06:13.144909 (+ 121us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:13.145183 (+ 274us) server_negotiation.cc:299] Negotiation successful
0923 15:06:13.145375 (+ 192us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260923 15:06:13.146661 4022 catalog_manager.cc:1475] Loading table and tablet metadata into memory...
I20260923 15:06:13.147544 4022 catalog_manager.cc:1484] Initializing Kudu cluster ID...
I20260923 15:06:13.148953 3907 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:13.135794 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48352 (local address 127.13.75.189:39767)
0923 15:06:13.135837 (+ 43us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:13.135839 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:13.137836 (+ 1997us) server_negotiation.cc:407] Connection header received
0923 15:06:13.137925 (+ 89us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:13.137926 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:13.137939 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:13.138004 (+ 65us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:13.138711 (+ 707us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:13.139579 (+ 868us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:13.140688 (+ 1109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:13.140985 (+ 297us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:13.141072 (+ 87us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:13.145572 (+ 4500us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:06:13.145574 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:06:13.145574 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:06:13.145584 (+ 10us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:06:13.147565 (+ 1981us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:13.148114 (+ 549us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:13.148116 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:13.148116 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:13.148196 (+ 80us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:13.148395 (+ 199us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:13.148396 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:13.148396 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:13.148537 (+ 141us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:06:13.148643 (+ 106us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:13.148755 (+ 112us) server_negotiation.cc:299] Negotiation successful
0923 15:06:13.148856 (+ 101us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":21}
I20260923 15:06:13.151304 3953 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 617b9a2ed1c44802be824f496c8144b6 [term 1 FOLLOWER]: Refusing update from remote peer 925e4dec28a444a0b61b5bcb77464ce1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 15:06:13.151664 4019 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 925e4dec28a444a0b61b5bcb77464ce1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "617b9a2ed1c44802be824f496c8144b6" member_type: VOTER last_known_addr { host: "127.13.75.188" port: 34245 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
I20260923 15:06:13.153254 3988 sys_catalog.cc:455] T 00000000000000000000000000000000 P 617b9a2ed1c44802be824f496c8144b6 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 925e4dec28a444a0b61b5bcb77464ce1. Latest consensus state: current_term: 1 leader_uuid: "925e4dec28a444a0b61b5bcb77464ce1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "925e4dec28a444a0b61b5bcb77464ce1" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 42059 } } peers { permanent_uuid: "33e89a99a9f146cb93c7445c741be846" member_type: VOTER last_known_addr { host: "127.13.75.189" port: 39767 } } peers { permanent_uuid: "617b9a2ed1c44802be824f496c8144b6" member_type: VOTER last_known_addr { host: "127.13.75.188" port: 34245 } } }
I20260923 15:06:13.153383 3988 sys_catalog.cc:458] T 00000000000000000000000000000000 P 617b9a2ed1c44802be824f496c8144b6 [sys.catalog]: This master's current role is: FOLLOWER
I20260923 15:06:13.154997 3985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:13.136232 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52844 (local address 127.13.75.188:34245)
0923 15:06:13.136280 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:13.136282 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:13.145457 (+ 9175us) server_negotiation.cc:407] Connection header received
0923 15:06:13.145578 (+ 121us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:13.145579 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:13.145596 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:13.145631 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:13.146947 (+ 1316us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:13.147762 (+ 815us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:13.148847 (+ 1085us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:13.149221 (+ 374us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:13.149274 (+ 53us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:13.151177 (+ 1903us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:06:13.151179 (+ 2us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:06:13.151180 (+ 1us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:06:13.151189 (+ 9us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:06:13.153126 (+ 1937us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:13.153866 (+ 740us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:13.153868 (+ 2us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:13.153868 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:13.153941 (+ 73us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:13.154238 (+ 297us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:13.154239 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:13.154239 (+ 0us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:13.154408 (+ 169us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:06:13.154524 (+ 116us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:13.154782 (+ 258us) server_negotiation.cc:299] Negotiation successful
0923 15:06:13.154901 (+ 119us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":19}
I20260923 15:06:13.160352 3994 sys_catalog.cc:455] T 00000000000000000000000000000000 P 925e4dec28a444a0b61b5bcb77464ce1 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "925e4dec28a444a0b61b5bcb77464ce1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "925e4dec28a444a0b61b5bcb77464ce1" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 42059 } } peers { permanent_uuid: "33e89a99a9f146cb93c7445c741be846" member_type: VOTER last_known_addr { host: "127.13.75.189" port: 39767 } } peers { permanent_uuid: "617b9a2ed1c44802be824f496c8144b6" member_type: VOTER last_known_addr { host: "127.13.75.188" port: 34245 } } }
I20260923 15:06:13.160478 3994 sys_catalog.cc:458] T 00000000000000000000000000000000 P 925e4dec28a444a0b61b5bcb77464ce1 [sys.catalog]: This master's current role is: LEADER
I20260923 15:06:13.160414 3988 sys_catalog.cc:455] T 00000000000000000000000000000000 P 617b9a2ed1c44802be824f496c8144b6 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "925e4dec28a444a0b61b5bcb77464ce1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "925e4dec28a444a0b61b5bcb77464ce1" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 42059 } } peers { permanent_uuid: "33e89a99a9f146cb93c7445c741be846" member_type: VOTER last_known_addr { host: "127.13.75.189" port: 39767 } } peers { permanent_uuid: "617b9a2ed1c44802be824f496c8144b6" member_type: VOTER last_known_addr { host: "127.13.75.188" port: 34245 } } }
I20260923 15:06:13.160521 3988 sys_catalog.cc:458] T 00000000000000000000000000000000 P 617b9a2ed1c44802be824f496c8144b6 [sys.catalog]: This master's current role is: FOLLOWER
I20260923 15:06:13.160677 3903 consensus_peers.cc:656] Retrying to get permanent uuid for remote peer: member_type: VOTER last_known_addr { host: "127.13.75.188" port: 34245 } attempt: 3
I20260923 15:06:13.160754 4024 mvcc.cc:205] Tried to move back new op lower bound from 7332560786024161280 to 7332560785967337472. Current Snapshot: MvccSnapshot[applied={T|T < 7332560786024161280}]
Sep 23 15:06:13 dist-test-slave-cnb0 krb5kdc[3747](info): TGS_REQ (2 etypes {17 16}) 127.0.0.1: ISSUE: authtime 1790175972, etypes {rep=17 tkt=17 ses=17}, kudu/127.13.75.189@KRBTEST.COM for kudu/127.13.75.188@KRBTEST.COM
I20260923 15:06:13.167451 4025 mvcc.cc:205] Tried to move back new op lower bound from 7332560786024161280 to 7332560785967337472. Current Snapshot: MvccSnapshot[applied={T|T < 7332560786024161280}]
I20260923 15:06:13.170753 3985 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:13.161079 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52846 (local address 127.13.75.188:34245)
0923 15:06:13.161138 (+ 59us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:13.161140 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:13.161151 (+ 11us) server_negotiation.cc:407] Connection header received
0923 15:06:13.161192 (+ 41us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:13.161192 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:13.161202 (+ 10us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:13.161269 (+ 67us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:13.162295 (+ 1026us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:13.163161 (+ 866us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:13.164170 (+ 1009us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:13.164516 (+ 346us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:13.164595 (+ 79us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:13.166990 (+ 2395us) server_negotiation.cc:365] Received SASL_INITIATE NegotiatePB request
0923 15:06:13.166991 (+ 1us) server_negotiation.cc:881] Received SASL_INITIATE request from client
0923 15:06:13.166991 (+ 0us) server_negotiation.cc:892] Client requested to use mechanism: GSSAPI
0923 15:06:13.166999 (+ 8us) server_negotiation.cc:922] calling sasl_server_start()
0923 15:06:13.168856 (+ 1857us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:13.169286 (+ 430us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:13.169287 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:13.169288 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:13.169394 (+ 106us) server_negotiation.cc:377] Sending SASL_CHALLENGE NegotiatePB response
0923 15:06:13.170123 (+ 729us) server_negotiation.cc:365] Received SASL_RESPONSE NegotiatePB request
0923 15:06:13.170124 (+ 1us) server_negotiation.cc:956] Received SASL_RESPONSE request from client
0923 15:06:13.170125 (+ 1us) server_negotiation.cc:967] calling sasl_server_step()
0923 15:06:13.170279 (+ 154us) server_negotiation.cc:377] Sending SASL_SUCCESS NegotiatePB response
0923 15:06:13.170344 (+ 65us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:13.170544 (+ 200us) server_negotiation.cc:299] Negotiation successful
0923 15:06:13.170644 (+ 100us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":37}
I20260923 15:06:13.170758 3905 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:13.160971 (+ 0us) reactor.cc:625] Submitting negotiation task for client connection to 127.13.75.188:34245 (local address 127.0.0.1:52846)
0923 15:06:13.161079 (+ 108us) negotiation.cc:107] Waiting for socket to connect
0923 15:06:13.161092 (+ 13us) client_negotiation.cc:174] Beginning negotiation
0923 15:06:13.161139 (+ 47us) client_negotiation.cc:252] Sending NEGOTIATE NegotiatePB request
0923 15:06:13.161786 (+ 647us) client_negotiation.cc:271] Received NEGOTIATE NegotiatePB response
0923 15:06:13.161787 (+ 1us) client_negotiation.cc:394] Received NEGOTIATE response from server
0923 15:06:13.161992 (+ 205us) client_negotiation.cc:189] Negotiated authn=SASL
0923 15:06:13.162048 (+ 56us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:13.162051 (+ 3us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:13.163240 (+ 1189us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:13.163242 (+ 2us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:13.164073 (+ 831us) client_negotiation.cc:514] Sending TLS_HANDSHAKE message to server
0923 15:06:13.164075 (+ 2us) client_negotiation.cc:252] Sending TLS_HANDSHAKE NegotiatePB request
0923 15:06:13.164600 (+ 525us) client_negotiation.cc:271] Received TLS_HANDSHAKE NegotiatePB response
0923 15:06:13.164601 (+ 1us) client_negotiation.cc:527] Received TLS_HANDSHAKE response from server
0923 15:06:13.164678 (+ 77us) client_negotiation.cc:548] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:13.165209 (+ 531us) client_negotiation.cc:623] Initiating SASL GSSAPI handshake
0923 15:06:13.165215 (+ 6us) client_negotiation.cc:656] calling sasl_client_start()
0923 15:06:13.166867 (+ 1652us) client_negotiation.cc:252] Sending SASL_INITIATE NegotiatePB request
0923 15:06:13.168947 (+ 2080us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:06:13.168948 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:06:13.168949 (+ 1us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:06:13.169199 (+ 250us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:06:13.169916 (+ 717us) client_negotiation.cc:271] Received SASL_CHALLENGE NegotiatePB response
0923 15:06:13.169917 (+ 1us) client_negotiation.cc:694] Received SASL_CHALLENGE response from server
0923 15:06:13.169917 (+ 0us) client_negotiation.cc:760] calling sasl_client_step()
0923 15:06:13.169990 (+ 73us) client_negotiation.cc:252] Sending SASL_RESPONSE NegotiatePB request
0923 15:06:13.170374 (+ 384us) client_negotiation.cc:271] Received SASL_SUCCESS NegotiatePB response
0923 15:06:13.170375 (+ 1us) client_negotiation.cc:715] Received SASL_SUCCESS response from server
0923 15:06:13.170427 (+ 52us) client_negotiation.cc:769] Sending connection context
0923 15:06:13.170505 (+ 78us) client_negotiation.cc:240] Negotiation successful
0923 15:06:13.170622 (+ 117us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"client-negotiator.queue_time_us":84}
I20260923 15:06:13.171545 4022 catalog_manager.cc:1347] Generated new cluster ID: c3d35dc424374e739c875e087b8873e7
I20260923 15:06:13.171610 4022 catalog_manager.cc:1495] Initializing Kudu internal certificate authority...
I20260923 15:06:13.172422 3903 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 33e89a99a9f146cb93c7445c741be846: Bootstrap starting.
I20260923 15:06:13.173161 3903 tablet_bootstrap.cc:654] T 00000000000000000000000000000000 P 33e89a99a9f146cb93c7445c741be846: Neither blocks nor log segments found. Creating new log.
I20260923 15:06:13.173465 3903 log.cc:824] T 00000000000000000000000000000000 P 33e89a99a9f146cb93c7445c741be846: Log is configured to *not* fsync() on all Append() calls
I20260923 15:06:13.174798 3903 tablet_bootstrap.cc:492] T 00000000000000000000000000000000 P 33e89a99a9f146cb93c7445c741be846: No bootstrap required, opened a new log
I20260923 15:06:13.176961 3903 raft_consensus.cc:348] T 00000000000000000000000000000000 P 33e89a99a9f146cb93c7445c741be846 [term 0 FOLLOWER]: Replica starting. Triggering 0 pending ops. Active config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "925e4dec28a444a0b61b5bcb77464ce1" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 42059 } } peers { permanent_uuid: "33e89a99a9f146cb93c7445c741be846" member_type: VOTER last_known_addr { host: "127.13.75.189" port: 39767 } } peers { permanent_uuid: "617b9a2ed1c44802be824f496c8144b6" member_type: VOTER last_known_addr { host: "127.13.75.188" port: 34245 } }
I20260923 15:06:13.177120 3903 raft_consensus.cc:729] T 00000000000000000000000000000000 P 33e89a99a9f146cb93c7445c741be846 [term 0 FOLLOWER]: Becoming Follower/Learner. State: Replica: 33e89a99a9f146cb93c7445c741be846, State: Initialized, Role: FOLLOWER
I20260923 15:06:13.177208 3903 consensus_queue.cc:260] T 00000000000000000000000000000000 P 33e89a99a9f146cb93c7445c741be846 [NON_LEADER]: Queue going to NON_LEADER mode. State: All replicated index: 0, Majority replicated index: 0, Committed index: 0, Last appended: 0.0, Last appended by leader: 0, Current term: 0, Majority size: -1, State: 0, Mode: NON_LEADER, active raft config: opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "925e4dec28a444a0b61b5bcb77464ce1" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 42059 } } peers { permanent_uuid: "33e89a99a9f146cb93c7445c741be846" member_type: VOTER last_known_addr { host: "127.13.75.189" port: 39767 } } peers { permanent_uuid: "617b9a2ed1c44802be824f496c8144b6" member_type: VOTER last_known_addr { host: "127.13.75.188" port: 34245 } }
I20260923 15:06:13.177564 3903 sys_catalog.cc:564] T 00000000000000000000000000000000 P 33e89a99a9f146cb93c7445c741be846 [sys.catalog]: configured and running, proceeding with master startup.
I20260923 15:06:13.177637 3877 tablet_service.cc:1812] Received RequestConsensusVote() RPC: tablet_id: "00000000000000000000000000000000" candidate_uuid: "925e4dec28a444a0b61b5bcb77464ce1" candidate_term: 1 candidate_status { last_received { term: 0 index: 0 } } ignore_live_leader: true dest_uuid: "33e89a99a9f146cb93c7445c741be846"
I20260923 15:06:13.177770 3877 raft_consensus.cc:3037] T 00000000000000000000000000000000 P 33e89a99a9f146cb93c7445c741be846 [term 0 FOLLOWER]: Advancing to term 1
I20260923 15:06:13.178732 3877 raft_consensus.cc:2454] T 00000000000000000000000000000000 P 33e89a99a9f146cb93c7445c741be846 [term 1 FOLLOWER]: Leader election vote request: Granting yes vote for candidate 925e4dec28a444a0b61b5bcb77464ce1 in term 1.
I20260923 15:06:13.179124 3876 raft_consensus.cc:1261] T 00000000000000000000000000000000 P 33e89a99a9f146cb93c7445c741be846 [term 1 FOLLOWER]: Refusing update from remote peer 925e4dec28a444a0b61b5bcb77464ce1: Log matching property violated. Preceding OpId in replica: term: 0 index: 0. Preceding OpId from leader: term: 1 index: 2. (index mismatch)
I20260923 15:06:13.177568 4030 sys_catalog.cc:455] T 00000000000000000000000000000000 P 33e89a99a9f146cb93c7445c741be846 [sys.catalog]: SysCatalogTable state changed. Reason: RaftConsensus started. Latest consensus state: current_term: 0 committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "925e4dec28a444a0b61b5bcb77464ce1" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 42059 } } peers { permanent_uuid: "33e89a99a9f146cb93c7445c741be846" member_type: VOTER last_known_addr { host: "127.13.75.189" port: 39767 } } peers { permanent_uuid: "617b9a2ed1c44802be824f496c8144b6" member_type: VOTER last_known_addr { host: "127.13.75.188" port: 34245 } } }
I20260923 15:06:13.179461 4030 sys_catalog.cc:458] T 00000000000000000000000000000000 P 33e89a99a9f146cb93c7445c741be846 [sys.catalog]: This master's current role is: FOLLOWER
I20260923 15:06:13.179535 4019 consensus_queue.cc:1035] T 00000000000000000000000000000000 P 925e4dec28a444a0b61b5bcb77464ce1 [LEADER]: Connected to new peer: Peer: permanent_uuid: "33e89a99a9f146cb93c7445c741be846" member_type: VOTER last_known_addr { host: "127.13.75.189" port: 39767 }, Status: LMP_MISMATCH, Last received: 0.0, Next index: 1, Last known committed idx: 0, Time since last communication: 0.000s
W20260923 15:06:13.180011 4041 catalog_manager.cc:1558] T 00000000000000000000000000000000 P 33e89a99a9f146cb93c7445c741be846: loading cluster ID for follower catalog manager: Not found: cluster ID entry not found
W20260923 15:06:13.180069 4041 catalog_manager.cc:873] Not found: cluster ID entry not found: failed to prepare follower catalog manager, will retry
I20260923 15:06:13.181677 4030 sys_catalog.cc:455] T 00000000000000000000000000000000 P 33e89a99a9f146cb93c7445c741be846 [sys.catalog]: SysCatalogTable state changed. Reason: New leader 925e4dec28a444a0b61b5bcb77464ce1. Latest consensus state: current_term: 1 leader_uuid: "925e4dec28a444a0b61b5bcb77464ce1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "925e4dec28a444a0b61b5bcb77464ce1" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 42059 } } peers { permanent_uuid: "33e89a99a9f146cb93c7445c741be846" member_type: VOTER last_known_addr { host: "127.13.75.189" port: 39767 } } peers { permanent_uuid: "617b9a2ed1c44802be824f496c8144b6" member_type: VOTER last_known_addr { host: "127.13.75.188" port: 34245 } } }
I20260923 15:06:13.181813 4030 sys_catalog.cc:458] T 00000000000000000000000000000000 P 33e89a99a9f146cb93c7445c741be846 [sys.catalog]: This master's current role is: FOLLOWER
I20260923 15:06:13.181890 4030 sys_catalog.cc:455] T 00000000000000000000000000000000 P 33e89a99a9f146cb93c7445c741be846 [sys.catalog]: SysCatalogTable state changed. Reason: Replicated consensus-only round. Latest consensus state: current_term: 1 leader_uuid: "925e4dec28a444a0b61b5bcb77464ce1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "925e4dec28a444a0b61b5bcb77464ce1" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 42059 } } peers { permanent_uuid: "33e89a99a9f146cb93c7445c741be846" member_type: VOTER last_known_addr { host: "127.13.75.189" port: 39767 } } peers { permanent_uuid: "617b9a2ed1c44802be824f496c8144b6" member_type: VOTER last_known_addr { host: "127.13.75.188" port: 34245 } } }
I20260923 15:06:13.181982 4030 sys_catalog.cc:458] T 00000000000000000000000000000000 P 33e89a99a9f146cb93c7445c741be846 [sys.catalog]: This master's current role is: FOLLOWER
I20260923 15:06:13.183336 4043 mvcc.cc:205] Tried to move back new op lower bound from 7332560786024161280 to 7332560785967337472. Current Snapshot: MvccSnapshot[applied={T|T < 7332560786024161280}]
I20260923 15:06:13.200726 4019 sys_catalog.cc:455] T 00000000000000000000000000000000 P 925e4dec28a444a0b61b5bcb77464ce1 [sys.catalog]: SysCatalogTable state changed. Reason: Peer health change. Latest consensus state: current_term: 1 leader_uuid: "925e4dec28a444a0b61b5bcb77464ce1" committed_config { opid_index: -1 OBSOLETE_local: false peers { permanent_uuid: "925e4dec28a444a0b61b5bcb77464ce1" member_type: VOTER last_known_addr { host: "127.13.75.190" port: 42059 } } peers { permanent_uuid: "33e89a99a9f146cb93c7445c741be846" member_type: VOTER last_known_addr { host: "127.13.75.189" port: 39767 } } peers { permanent_uuid: "617b9a2ed1c44802be824f496c8144b6" member_type: VOTER last_known_addr { host: "127.13.75.188" port: 34245 } } }
I20260923 15:06:13.200866 4019 sys_catalog.cc:458] T 00000000000000000000000000000000 P 925e4dec28a444a0b61b5bcb77464ce1 [sys.catalog]: This master's current role is: LEADER
I20260923 15:06:13.203469 4022 catalog_manager.cc:1370] Generated new certificate authority record
I20260923 15:06:13.204463 4022 catalog_manager.cc:1504] Loading token signing keys...
I20260923 15:06:13.216646 4022 catalog_manager.cc:5868] T 00000000000000000000000000000000 P 925e4dec28a444a0b61b5bcb77464ce1: Generated new TSK 0
I20260923 15:06:13.216900 4022 catalog_manager.cc:1514] Initializing in-progress tserver states...
W20260923 15:06:13.243582 3907 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 15:06:13.242004 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48362 (local address 127.13.75.189:39767)
0923 15:06:13.242056 (+ 52us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:13.242059 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:13.242813 (+ 754us) server_negotiation.cc:407] Connection header received
0923 15:06:13.242972 (+ 159us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:13.242973 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:13.242985 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:13.243016 (+ 31us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:13.243500 (+ 484us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48362: BlockingRecv error: recv got EOF from 127.0.0.1:48362 (error 108)
Metrics: {"server-negotiator.queue_time_us":27}
W20260923 15:06:13.244372 3985 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 15:06:13.242441 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52856 (local address 127.13.75.188:34245)
0923 15:06:13.242482 (+ 41us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:13.242484 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:13.243845 (+ 1361us) server_negotiation.cc:407] Connection header received
0923 15:06:13.243946 (+ 101us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:13.243947 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:13.243959 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:13.243989 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:13.244310 (+ 321us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52856: BlockingRecv error: recv got EOF from 127.0.0.1:52856 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260923 15:06:13.245543 3832 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 15:06:13.244724 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52070 (local address 127.13.75.190:42059)
0923 15:06:13.244766 (+ 42us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:13.244768 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:13.244982 (+ 214us) server_negotiation.cc:407] Connection header received
0923 15:06:13.245124 (+ 142us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:13.245126 (+ 2us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:13.245137 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:13.245167 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:13.245477 (+ 310us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:52070: BlockingRecv error: recv got EOF from 127.0.0.1:52070 (error 108)
Metrics: {"server-negotiator.queue_time_us":23}
W20260923 15:06:13.248239 3907 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 15:06:13.247360 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48366 (local address 127.13.75.189:39767)
0923 15:06:13.247408 (+ 48us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:13.247410 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:13.247659 (+ 249us) server_negotiation.cc:407] Connection header received
0923 15:06:13.247806 (+ 147us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:13.247807 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:13.247828 (+ 21us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:13.247858 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:13.248187 (+ 329us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48366: BlockingRecv error: recv got EOF from 127.0.0.1:48366 (error 108)
Metrics: {"server-negotiator.queue_time_us":28}
W20260923 15:06:13.251453 3907 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 15:06:13.250596 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48382 (local address 127.13.75.189:39767)
0923 15:06:13.250641 (+ 45us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:13.250643 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:13.250932 (+ 289us) server_negotiation.cc:407] Connection header received
0923 15:06:13.251035 (+ 103us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:13.251036 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:13.251047 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:13.251077 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:13.251397 (+ 320us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48382: BlockingRecv error: recv got EOF from 127.0.0.1:48382 (error 108)
Metrics: {"server-negotiator.queue_time_us":27}
W20260923 15:06:13.255342 3907 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 15:06:13.254528 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48386 (local address 127.13.75.189:39767)
0923 15:06:13.254562 (+ 34us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:13.254564 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:13.254815 (+ 251us) server_negotiation.cc:407] Connection header received
0923 15:06:13.254938 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:13.254939 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:13.254950 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:13.254980 (+ 30us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:13.255292 (+ 312us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48386: BlockingRecv error: recv got EOF from 127.0.0.1:48386 (error 108)
Metrics: {"server-negotiator.queue_time_us":17}
W20260923 15:06:13.261319 3907 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 15:06:13.260465 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48390 (local address 127.13.75.189:39767)
0923 15:06:13.260505 (+ 40us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:13.260507 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:13.260790 (+ 283us) server_negotiation.cc:407] Connection header received
0923 15:06:13.260884 (+ 94us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:13.260885 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:13.260896 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:13.260924 (+ 28us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:13.261260 (+ 336us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48390: BlockingRecv error: recv got EOF from 127.0.0.1:48390 (error 108)
Metrics: {"server-negotiator.queue_time_us":20}
W20260923 15:06:13.271641 3907 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 15:06:13.270576 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48396 (local address 127.13.75.189:39767)
0923 15:06:13.270636 (+ 60us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:13.270638 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:13.271025 (+ 387us) server_negotiation.cc:407] Connection header received
0923 15:06:13.271140 (+ 115us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:13.271141 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:13.271153 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:13.271185 (+ 32us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:13.271566 (+ 381us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48396: BlockingRecv error: recv got EOF from 127.0.0.1:48396 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260923 15:06:13.290227 3907 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 15:06:13.289181 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48398 (local address 127.13.75.189:39767)
0923 15:06:13.289227 (+ 46us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:13.289229 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:13.289577 (+ 348us) server_negotiation.cc:407] Connection header received
0923 15:06:13.289700 (+ 123us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:13.289701 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:13.289715 (+ 14us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:13.289748 (+ 33us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:13.290164 (+ 416us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48398: BlockingRecv error: recv got EOF from 127.0.0.1:48398 (error 108)
Metrics: {"server-negotiator.queue_time_us":22}
W20260923 15:06:13.337044 3907 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 15:06:13.335945 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48414 (local address 127.13.75.189:39767)
0923 15:06:13.335999 (+ 54us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:13.336001 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:13.336360 (+ 359us) server_negotiation.cc:407] Connection header received
0923 15:06:13.336485 (+ 125us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:13.336486 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:13.336499 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:13.336534 (+ 35us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:13.336977 (+ 443us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48414: BlockingRecv error: recv got EOF from 127.0.0.1:48414 (error 108)
Metrics: {"server-negotiator.queue_time_us":24}
W20260923 15:06:13.403586 3907 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 15:06:13.402687 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48428 (local address 127.13.75.189:39767)
0923 15:06:13.402780 (+ 93us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:13.402783 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:13.402872 (+ 89us) server_negotiation.cc:407] Connection header received
0923 15:06:13.403005 (+ 133us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:13.403006 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:13.403023 (+ 17us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:13.403123 (+ 100us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:13.403505 (+ 382us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48428: BlockingRecv error: recv got EOF from 127.0.0.1:48428 (error 108)
Metrics: {"server-negotiator.queue_time_us":59}
W20260923 15:06:13.533674 3907 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 15:06:13.532999 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48436 (local address 127.13.75.189:39767)
0923 15:06:13.533077 (+ 78us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:13.533079 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:13.533124 (+ 45us) server_negotiation.cc:407] Connection header received
0923 15:06:13.533199 (+ 75us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:13.533200 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:13.533211 (+ 11us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:13.533266 (+ 55us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:13.533601 (+ 335us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48436: BlockingRecv error: recv got EOF from 127.0.0.1:48436 (error 108)
Metrics: {"server-negotiator.queue_time_us":45}
W20260923 15:06:13.791966 3907 negotiation.cc:337] Failed RPC negotiation. Trace:
0923 15:06:13.791259 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48438 (local address 127.13.75.189:39767)
0923 15:06:13.791354 (+ 95us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:13.791356 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:13.791371 (+ 15us) server_negotiation.cc:407] Connection header received
0923 15:06:13.791407 (+ 36us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:13.791408 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:13.791421 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:13.791510 (+ 89us) server_negotiation.cc:226] Negotiated authn=SASL
0923 15:06:13.791912 (+ 402us) negotiation.cc:327] Negotiation complete: Network error: Server connection negotiation failed: server connection from 127.0.0.1:48438: BlockingRecv error: recv got EOF from 127.0.0.1:48438 (error 108)
Metrics: {"server-negotiator.queue_time_us":64}
I20260923 15:06:14.137072 4000 catalog_manager.cc:1259] Loaded cluster ID: c3d35dc424374e739c875e087b8873e7
I20260923 15:06:14.137203 4000 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 617b9a2ed1c44802be824f496c8144b6: loading cluster ID for follower catalog manager: success
I20260923 15:06:14.138125 4000 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 617b9a2ed1c44802be824f496c8144b6: acquiring CA information for follower catalog manager: success
I20260923 15:06:14.138491 4000 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 617b9a2ed1c44802be824f496c8144b6: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260923 15:06:14.181244 4041 catalog_manager.cc:1259] Loaded cluster ID: c3d35dc424374e739c875e087b8873e7
I20260923 15:06:14.181350 4041 catalog_manager.cc:1552] T 00000000000000000000000000000000 P 33e89a99a9f146cb93c7445c741be846: loading cluster ID for follower catalog manager: success
I20260923 15:06:14.182161 4041 catalog_manager.cc:1574] T 00000000000000000000000000000000 P 33e89a99a9f146cb93c7445c741be846: acquiring CA information for follower catalog manager: success
I20260923 15:06:14.182485 4041 catalog_manager.cc:1602] T 00000000000000000000000000000000 P 33e89a99a9f146cb93c7445c741be846: importing token verification keys for follower catalog manager: success; most recent TSK sequence number 0
I20260923 15:06:14.308779 4111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:14.305569 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48450 (local address 127.13.75.189:39767)
0923 15:06:14.305744 (+ 175us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:14.305747 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:14.305823 (+ 76us) server_negotiation.cc:407] Connection header received
0923 15:06:14.305933 (+ 110us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:14.305934 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:14.306005 (+ 71us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:14.306105 (+ 100us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 15:06:14.306297 (+ 192us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:14.306959 (+ 662us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:14.307834 (+ 875us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:14.308080 (+ 246us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:14.308163 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:14.308362 (+ 199us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 15:06:14.308493 (+ 131us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 15:06:14.308598 (+ 105us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:14.308644 (+ 46us) server_negotiation.cc:299] Negotiation successful
0923 15:06:14.308670 (+ 26us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":149,"thread_start_us":91,"threads_started":1}
I20260923 15:06:14.313836 4117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:14.311080 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52860 (local address 127.13.75.188:34245)
0923 15:06:14.311199 (+ 119us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:14.311201 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:14.311212 (+ 11us) server_negotiation.cc:407] Connection header received
0923 15:06:14.311298 (+ 86us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:14.311299 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:14.311357 (+ 58us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:14.311452 (+ 95us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 15:06:14.311561 (+ 109us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:14.312103 (+ 542us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:14.312925 (+ 822us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:14.313173 (+ 248us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:14.313256 (+ 83us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:14.313447 (+ 191us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 15:06:14.313579 (+ 132us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 15:06:14.313682 (+ 103us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:14.313703 (+ 21us) server_negotiation.cc:299] Negotiation successful
0923 15:06:14.313728 (+ 25us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":93,"thread_start_us":62,"threads_started":1}
I20260923 15:06:14.319185 4124 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:14.315779 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52072 (local address 127.13.75.190:42059)
0923 15:06:14.315911 (+ 132us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:14.315914 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:14.315922 (+ 8us) server_negotiation.cc:407] Connection header received
0923 15:06:14.315974 (+ 52us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:14.315974 (+ 0us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:14.316035 (+ 61us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:14.316151 (+ 116us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 15:06:14.316265 (+ 114us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:14.316877 (+ 612us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:14.317975 (+ 1098us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:14.318363 (+ 388us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:14.318430 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:14.318624 (+ 194us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 15:06:14.318758 (+ 134us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 15:06:14.318826 (+ 68us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:14.319044 (+ 218us) server_negotiation.cc:299] Negotiation successful
0923 15:06:14.319077 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":105,"thread_start_us":66,"threads_started":1}
I20260923 15:06:14.319304 4111 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:14.315722 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:48466 (local address 127.13.75.189:39767)
0923 15:06:14.315804 (+ 82us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:14.315807 (+ 3us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:14.315977 (+ 170us) server_negotiation.cc:407] Connection header received
0923 15:06:14.316076 (+ 99us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:14.316077 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:14.316090 (+ 13us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:14.316148 (+ 58us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 15:06:14.316339 (+ 191us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:14.317235 (+ 896us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:14.318274 (+ 1039us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:14.318626 (+ 352us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:14.318711 (+ 85us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:14.318932 (+ 221us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 15:06:14.318972 (+ 40us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 15:06:14.319024 (+ 52us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:14.319186 (+ 162us) server_negotiation.cc:299] Negotiation successful
0923 15:06:14.319220 (+ 34us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":54}
I20260923 15:06:14.319327 4117 negotiation.cc:339] RPC negotiation tracing enabled. Trace:
0923 15:06:14.315745 (+ 0us) reactor.cc:625] Submitting negotiation task for server connection from 127.0.0.1:52862 (local address 127.13.75.188:34245)
0923 15:06:14.315794 (+ 49us) server_negotiation.cc:206] Beginning negotiation
0923 15:06:14.315796 (+ 2us) server_negotiation.cc:399] Waiting for connection header
0923 15:06:14.316108 (+ 312us) server_negotiation.cc:407] Connection header received
0923 15:06:14.316136 (+ 28us) server_negotiation.cc:365] Received NEGOTIATE NegotiatePB request
0923 15:06:14.316137 (+ 1us) server_negotiation.cc:461] Received NEGOTIATE request from client
0923 15:06:14.316149 (+ 12us) server_negotiation.cc:377] Sending NEGOTIATE NegotiatePB response
0923 15:06:14.316200 (+ 51us) server_negotiation.cc:226] Negotiated authn=TOKEN
0923 15:06:14.316423 (+ 223us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:14.317229 (+ 806us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:14.318355 (+ 1126us) server_negotiation.cc:365] Received TLS_HANDSHAKE NegotiatePB request
0923 15:06:14.318665 (+ 310us) server_negotiation.cc:377] Sending TLS_HANDSHAKE NegotiatePB response
0923 15:06:14.318732 (+ 67us) server_negotiation.cc:657] Negotiated auth-only TLSv1.2 with cipher ECDHE-RSA-AES128-GCM-SHA256 TLSv1.2 Kx=ECDH Au=RSA Enc=AESGCM(128) Mac=AEAD
0923 15:06:14.318915 (+ 183us) server_negotiation.cc:365] Received TOKEN_EXCHANGE NegotiatePB request
0923 15:06:14.318948 (+ 33us) server_negotiation.cc:377] Sending TOKEN_EXCHANGE NegotiatePB response
0923 15:06:14.319007 (+ 59us) server_negotiation.cc:1035] Waiting for connection context
0923 15:06:14.319115 (+ 108us) server_negotiation.cc:299] Negotiation successful
0923 15:06:14.319148 (+ 33us) negotiation.cc:327] Negotiation complete: OK
Metrics: {"server-negotiator.queue_time_us":26}
I20260923 15:06:14.321231 13614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKC4soy/build/release/bin/kudu with pid 3760
I20260923 15:06:14.326542 13614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKC4soy/build/release/bin/kudu with pid 3835
I20260923 15:06:14.331956 13614 external_mini_cluster.cc:1595] Killing /tmp/dist-test-taskKC4soy/build/release/bin/kudu with pid 3910
2026-09-23T15:06:14Z chronyd exiting
[ OK ] ConnectToFollowerMasterTest.AuthnTokenVerifierHaveKeys/1 (2879 ms)
[----------] 2 tests from ConnectToFollowerMasterTest (5698 ms total)
[----------] Global test environment tear-down
[==========] 56 tests from 4 test suites ran. (282847 ms total)
[ PASSED ] 55 tests.
[ FAILED ] 1 test, listed below:
[ FAILED ] SecurityITest.TestJwtMiniCluster
1 FAILED TEST